AI and Low Code

Preparing your business for this new world

We no longer require the technical knowledge to connect complex systems together, but without appropriate training, this could sink a business.

My views on AI are mostly in the 'love' category, and for the record, I am a very heavy user of AI; whether that's ChatGPT, Replit, or our 'paranoia AI' lab (Where we run local air-gapped sensitive AI tasks). Perhaps in this sense I sit apart from many of my technically minded security peers, who on the whole tend to dislike and oppose AI more heavily than they like it.

I strongly believe that those who don't start using AI appropriately in the business world will be at a disadvantage, either as an employee, or a business entity.

Yes, I do worry about what AI is doing to certain jobs and industries, and what does the future look like for the next generation. But in the short term, I mostly about the misunderstanding of AI and its reliability in the business world.

Regardless of our independent views on AI, we all need to acknowledge:

AI is likely the most transformative tool the world has seen in many decades.
It is here to stay. Bubble, Boom, or otherwise, it'll remain.

AI is becoming far more accessible, and being increasingly pushed as a tool that anyone to pick up and use effectively.

In the general-public space, this usage tends to look like my dad downloading chatGPT onto his phone and typing in “New Computer”, then getting annoyed that it doesn't show him a list of computers that suit his needs.

The irony being that he'll go to Google's traditional search engine and type in “provide me with a new computer that meets my personal life needs”, and complain he gets blog posts instead of store results.

In the business space, we are using AI increasingly more expansively across business functions, commonly tools like Claude or ChatGPT are issued across the majority of the company on enterprise licences, allowing most employees to access the huge benefits of LLMs with all the compliance boxes ticked around data processing.

But with this broad spectrum approach to employee AI access, has come a wave of non-technically trained (and i truly mean no insult here) employees using AI to process information critical to a business process, and accepting the output with no, or very little, review.

Here, we have introduced my sleepless nights; but first, I'd like to step back and talk about the most underrated and perhaps mis-understood pillar of IT security: data integrity.

Data Integrity

I stand firmly along side the 'CIA Triad'. 3 pillars, that if security teams uphold, (generally) keep a business safe. And despite security bodies trying to squash more and more letters onto the end of CIA (CIAUAP, CIAAN); I stand by the classic for its simplicity.

C = Confidentiality

is our data private? Are our passwords safe from prying eyes? Can the hacker see my documents?

A = Availability

Are our systems designed to keep running or recovery quickly? If our website goes down, how fast does it come back up?

I = Integrity

Can we trust our data to be correct?

Often 'correct' is about ensuring people don't accidentally (or purposely) fiddle with data between leaving place A, and arriving at place B. But in this case, we want to ensure a process or tool couldn't misinterpret our data.

AI and Data Integrity

AI in the form we currently refer to it as (an LLM) takes in words, interprets those words, processes them, predicts the next most common words, and spits out a response.

That processing of words does not come from any conscious thought, sensible thinking, or logical path; it comes from a knowledge base that suggests "this is the next most likely word in the answer to a question”, not because that word is correct, but because out of all of the data the AI has in its databank, and that in the context of your question, this particular set of words tends to be the most common answer.

I am heavily Heavily paraphrasing how AI works here.

It doesn't really process words but rather tokens; the process is close enough for the purpose of this article, but for great info on how an AI is really treating your words, see this article by Sean Trott.

'knowledge base' and 'the most common answer' is also a huge (but also close enough) oversimplification. A brilliant article on this topic that only slightly melts the mind can be found here, by Claire Longo.

This makes AI great at anything to do with language. It can interpret sentiment, understand the meaning of confusing questions; but it is not mathematically accurate.

If you ask an AI what 2+2 is, it'll tell you 4; not because it has worked out the answer, but because the data it's been given says 4 is the most common word that answers that question.

Where the scary stuff starts.

Now that we understand that AI is just predicting answers, and interpreting questions based on the data it has, let's talk about the problem.

Low code / no code automation tools (Zapier, N8N, Make) are getting rapidly adopted by businesses thanks to their ability to connect two systems together, and then passed to employees without IT or Developer training; with claims of the complexity being reduced to near-zero.

These tools are brilliant; and I don't want to discourage their usage; The ability to allow a majority of staff, instead of a minority, to action their own automations is an efficiency power house; but there must be a heavy set of guard rails implemented and solid training put in place to allow this freedom.

We are actively seeing more and more companies hook these tools up to critical systems to extract, transform with AI, and load that data into another critical system.

If a developer (or other person trained in computer science or similar) was to do this, data X would be taken out of system A; run through a mathematical function which creates the desired output (data Y), and that data loaded into System B.

This process, because it's written in code and uses mathematical steps, is:

Consistent
We put in Data X, we get out Data Y. We can run this code once, or a thousand times, it'll always be the same

Testable
As it will always run the same process every time, we can test that code and validate it; thus we know that process works, every time.

Logical
It follows machine based instructions; it interprets nothing, and follows hard rules. Which means….

Thus, it is reliable, and the data has integrity!
We can trust this data.

A developer can prove that their function takes in Data X, spits out Data Y, and it will always do so. Everyone can sleep well knowing that the company's financial data is being handled correctly.

Throughout this process A developer may or may not use lowcode/nocode solutions; that's neither here nor there; the core point here is there is a lack of AI, or rather, a purely logical computational process.

Until recently, we have had a barrier to entry; which was technical developer knowledge and training. You were required to know how IT systems work, how code and data works, and know how computer systems talk to each other works; before you could connect these systems.

This barrier to entry meant that, without the right knowledge to do this safely, you could not:

Connect System A to System B; and/or,
Transform the data from X to Y.

And so (generally speaking!), people were handling data safely.

But now:

Lowcode /no code solutions have removed the barrier to entry for point 1; and
AI has removed the barrier to entry for point 2.

So the above flow is increasingly often looking like this:

An employee without the appropriate training uses a low-code solution to connect System A to System B (this isn't such a bad thing in itself).
They take Data X from System A, feed it into chatGPT to transform or interpret the data, then feed what is hopefully Data Y it into System B.

This process, because it's using AIs predictive nature is:

Not logical
Data is interpreted; we don't really know what its answer will be, as it's a language based system.

Not consistent
If we cannot predict its answer, we cannot call it consistent, and without consistency the data transformation is...

Not testable
So the data it spits out cannot have the integrity needed to be trusted in a critical process.

So while we're sleeping, we have no idea if AI is taking an order for 1000 screws from System A, and entering it into System B as:

1000 screws
10.00 screws
Pigeon

We can put in spot checks, or human-in-the-loop checks; but none of these give the business the confidence and long term efficiency that a 'coded' solution can give.

We may find that 999 times it works, but the 1000th time it fails. That failure could be insignificant, but lest we forget the post office scandal, built upon so many tiny errors compounded into massive problems.

What's the fix here?

We should not see this as a reason to block AI and/or low-code automation tools in a business; But rather we should understand that previous barrier to entry that we had, held importance, and we should look to build an appropriate barrier for this new suite of tools.

This will vary from company to company; but here's some examples that you could consider:

Access is only granted after AI and Low Code onboarding sessions; Sessions designed and run by people with true expertise in the subject. A half day workshop pays dividends in the long run.
Simple guard rails to minimise the highest risk (perhaps, no write access to the company finance systems)
An AI / Automations team with solid expertise that review and approve workflows before they hit production
Have InfoSec review workflows that involve high/critical risk workflows
Centrally manage the credentials you issue to the business. Either by system owners, IT, an AI team, or InfoSec.

By putting these (or other) controls in place ensures that you get the most bang for buck for your business from AI and low code tools, while heavily reducing the risk that we've spoken about above.