The Hardest Lesson in Dealing with Generative AI – Saying No.

“What is the hardest part about building with generative AI?”

The above was a question I received while doing a recent Q&A, which gave me a bit of a pause. It’s not like the question is out of left field or requires a lot of deep thought, but it is a bit of a scene change from what we normally get. Typically, questions have come down to “how can I use GenAI”, “what advantages does this bring our organization”, “does this cover specific scenario ABC” and so on. There is usually very little thought put into the specifics or the mechanics behind development.

As the Head of the Innovation Lab at Zeidler Group, I’ve been fortunate to be at the forefront of generative AI and serve as a resource for implementation and strategy when it comes to our usage of the technology.  It’s been exciting since the onset of wider adoption of the technology and seeing successful pilots turn into full products for our clients and us. Part of this has involved sitting down with teams at other firms and dissecting how they think and how the technology is being used so we can best provide our services in the regulatory space.

To that end, I stand by what I said in the moment: The hardest part about building with generative AI is saying no.

What do I mean by No

When I say “no” in this context, it’s in the simplest sense; it’s not a callout to the Meghan Trainor song or the post-minimalism art of Bruce Nauman. It is simply putting a foot down and stopping a project before it goes any further. There isn’t some secret technologist or engineering definition that grants special wisdom only applicable to generative AI. It’s the same exact “no” a toddler uses when refusing to eat their broccoli.

That being said, if it’s a common definition, why do I consider it to be so hard?

Why is saying No so hard

Enthusiasm in every industry is at a fever pitch, with new products and releases consistently hitting the public zeitgeist. Every day when I open LinkedIn, I’m bombarded by endless pitches of the new hot technical solutions using Generative AI, and the endless opportunities in the space if you “just do XYZ”. There’s this ravenous belief and desire that if we just include additional Generative AI features, all the unsolved problems become solvable, and nothing is impossible. In a way, it’s reminiscent of the frenzies around big data, no-code, and web3 that captured industry’s consciousness, changing how firms operate and evolve with time.

To that end, no becomes a bit of a difficult word to say, since it is directly flying in the face of unbridled hope and optimism, and frankly paints a negative picture towards whatever point you are making. Being the only one not dancing at the party isn’t necessarily a bad thing, but it can feel incredibly awkward and confrontational if it isn’t done carefully and with proper thought.

When to say No

The specifics of when to say no is unfortunately where it gets rather opaque, and why I consider it to be the hardest part when building with Generative AI. As much as my data-driven heart would prefer it, I have yet to find a purely analytical or numerical method of deciding whether “no” is the appropriate answer. Instead, it’s a bit more of an art than a science where there are several factors and judgment calls that exist.

Generally, I like to break these factors into wide categories, where we first think outside the immediate ask and try to think about the larger picture of a project. For most projects, this can be split into three main components: 1) the technological limitations of LLMs when related to the project; 2) the end requirements of the project vis-à-vis accuracy, time, and cost; 3) understanding the “why” behind a project. This means that even projects that sound very similar at first glance may have radically different needs and have different levels of feasibility.

For example, a project that seeks to fully automate reviewing a document is going to vary significantly if it is reviewing the first draft versus reviewing a final draft for sign-off. Conversely, a task where a human is still involved in the loop can be radically different from one where the intent is to fully automate.  Having a good idea of all the factors can help shape making a key decision on whether something should happen, but saying no will still be an independent judgment call.

Knowing if it is a conditional or permanent No

Now, to make everything more confusing, sometimes a “No” is a “No for now”, or a “No, but”. It’s important to remember that technology and processes change over time, and the reality we see today is not necessarily the reality we have tomorrow. For example, before multimodal LLMs really hit the scene and changed ad-hoc image processing, the state of the art for graph & chart detection was around 50% accuracy. During this period, I had to say no to a lot of projects that we are easily saying yes to today. Being willing to revisit those older projects and identify if the “no” from last year still stands today was important for our growth and still is important for us today.

There are, of course, scenarios where the “no” is going to be permanent – generally when something clearly violates a regulation or goes against fundamental values of an organization. At the end of the day, having flexibility and independent judgment around what we mean when we say “no” is key and not something that should have hard and fast rules.

Is this a sign of a bigger shift

To cap this off, I want to end with what I think is the reality of what we are facing when it comes to dealing with generative AI: Pandora’s box is open, and the technology is out there. It would be shortsighted to claim that this new crop of technological innovations is going to go away and that we shouldn’t consider its application at all. However, having a good idea of when it does make sense to use the technology, and when it doesn’t make sense, is increasingly a critical skill. Saying no at the right time for the right reasons will continue to be incredibly important and will enable organizations to best adapt to and use this suite of emerging technology.

If this resonates, I’d be interested to hear how you’re deciding when to say “no” in your own AI initiatives. At Zeidler Group, these are exactly the kinds of conversations we’re having every day as firms navigate where generative AI truly adds value. Please get in touch.