AI Digest - Jan24
What matters each month
Someone I worked with on the UK COVID response once told me: if you are in a situation with no clue what to do, standing perfectly still won’t change anything. Taking even a small step in one direction, on the other hand, will give you new information about what the world is like over there. And that information might help you figure out the best route to take.
In the face of uncertainty about the impact of AI tools on our work, this newsletter is an attempt to take those little steps. Getting responses back, about where my thoughts are right or wrong, interesting or unhelpful, is the best part. Plus, I love the conversations that they generate.
A belated Happy 2024 and all the best for the year ahead!
James
If you only read one thing
Last month, whilst reading one of the Year Ahead pieces (Ben Evan’s annual presentation, reliably one of the best) one of my WhatsApp groups flashed up. “It seems to be mostly questions this year” read the message, and they weren’t wrong. This pattern was echoed across 2024 forecasts I saw in the following weeks and reflects the very real uncertainty that faces organisations this year. Nobody is quite sure about how the capabilities of AI will develop, and nobody knows how companies will fare integrating the tools we already have.
The challenge, however, is to avoid letting uncertainty lead you to the default choice of Do Nothing (making this choice consciously is another matter). In almost all future scenarios, the failure to do any preparatory work in the coming months will leave organisations with a gap in skills, experience and knowledge. In some cases, standing still whilst others start experimenting will be too big a deficit to overcome. Instead, ask yourself what can be done to increase your chance of having the right answer, at some point in the future, without spending too much money now. Azeem Azhar’s year ahead outlook captured the range of options to respond to uncertainty from “scenario thinking, portfolio approaches, flexibility and adaptability, [to] resilience, judgement and preparedness”. Some of which we wrote about this week too.
This is the crux of what we are helping organisations to do. Understand and quantify uncertainty about the future, and make decisions now that they won’t regret down the line.
Ben Evan’s Presentation. Azeem Azhar’s Outlook. Building Flexibility. Whether to Wait.
Contents
What Is GenAI Good For?
Experiments bearing fruit
Be more ambitious - a bicycle for the mind
Case studies, data and more!
How To Successfully Integrate GenAI With Existing Organisations?
Building trust
Customer Service might not be the easy answer
Employees’ interests might not align with companies’
Good work in government
Our Recent Work
Building flexibly - Models and Beauty Contests
Zooming Out
What does the end state look like?
How fast do we get there?
Around the world
Copyright and the New York Times Case
Learning More
Newsletters, User Guide, Model Comparisons
The Lighter Side
What Is GenAI Good For?
Better ways of interacting with and integrating GenAI tools. Anyone who has used ChatGPT knows that it makes mistakes. The challenge for software providers & users is to integrate tools into systems which reliably do good work, even if the underlying components are fallible. Just like we do with human workers. The most exciting developments are on these systems, where we are seeing improvements in: ease of access; input & output structures; process embedding. Each of these developments change the way that companies and users can actually get valuable work from AI models, without needing advances in the AI models themselves. Microsoft has added a new AI assistant key to the Windows keyboard, the first new key in several decades, putting AI assistants literally at our fingertips. Remember, AI powered tools and systems will only get better from here as best practice starts to emerge. Software User Interfaces. Avoiding Hallucinations. AI Conversation Types. AI Key on Keyboards.
Image: 6 Different Types of Conversations with AI Chatbots. Source.
Be more ambitious. Those reporting the most value gain from using AI for personal use share a common trait. They are using AI tools to open up more time for specialist work where they have expertise and to reach proficiency in new areas. It was once feared that e-bikes would make people lazy; instead, they enabled people to cycle more frequently and further. AI tools appear to be the same: they allow you to cover more ground. Automate routine work, like finding background information on potential customers, to free up time for working out how best to approach them. Unsure how to write code to do this? Use an AI tutor to teach you. Bicycle for the mind. Automate Sales Development.
Start building business cases. Watching and waiting might have been enough last year, but once competitors start announcing plans to use AI the pressure to keep up will be immense. Evidence of productivity gains is now deep and broad. Watch investor days this year - pressure will build for companies to show they are harnessing these benefits too. These case studies might help in building your own business case. Lawyers see gains. Stackoverflow survey: big wins for developers. AI Referees in baseball. PwC, Carlyle, Estée Lauder and more.
How To Successfully Integrate GenAI With Existing Organisations
Trust in technology is hard won and easily lost. The big story in the UK this month has been the Post Office Scandal, where a known faulty, digitised accounting system was relied upon to wrongly convict hundreds of postmasters. “How do we know we can trust it?” has just become the question for new, load bearing IT systems. Examples of organisations getting this right range from games platform Steam to the US Supreme Court and emphasise the importance of consultation, contextualisation and a steady, considered decision making process. In contrast, organisations who lie about their use of AI and are then caught out should expect to face a stern backlash, even when the AI use was somewhat indirect. Post Office Scandal. Steam. US Supreme Court. Getting Caught Out.
Employee interests can diverge from their employers’. Upton Sinclair once said “It is difficult to get a man to understand something, when his salary depends on his not understanding it.” For organisations seeking to drive adoption the quote couldn’t be more relevant. Do you bill by the hour? Are new grads partly “paid” in training which earns them a senior role in the future? Do employees believe that previous career paths still exist? Just as organisations are scenario planning, so are your best employees. These challenges likely fall upon HR/People teams in the first instance. Ensuring that they have the necessary technical knowhow to understand employees’ concerns feels a good investment. Upton Sinclair. Scenario Planning Careers in the Age of AI.
Customer Service is not the easy answer. 9 months ago, automated customer service bots seemed first in line for a GenAI boost. Enthusiasm has waned for “hard” reasons of data loss risk and arguably trickier reasons related to the user experience. How a company talks to its customer is often a key part of the brand experience and reason for buying. We simply aren’t there yet on designing experiences that users (or customers) love. Perhaps the best places for GenAI interaction with customers are where interactions are already heavily structured and robotic? For example, there is good progress in bid writing software and public sector procurement functions are far from ready for automated responses. Designing Customer Experiences. Bid Writing. Customer Service Bots Fail.
Good work in government continues apace!
The UK Government publishes a framework for the use of GenAI in delivery of services to the public. Link. Analysis.
Washington State shares detailed advice for educators on the full range of issues raised by GenAI in the classroom. Link.
Our Recent Work
For technical projects, avoiding being locked in to any one choice is a necessary starting point in a quickly evolving world. Models and Beauty Contests.
Zooming Out
There are two big questions for the world of work:
As AI tools are adopted, what does the final distribution look like?
How fast do we get there?
In a world where huge swathes of professional work can be done for zero cost, who are the winners and losers? There are parallels to be drawn from previous revolutions where previously costly services like transport and communication plummeted in price. Behind predictions of new jobs emerging and limited overall job losses lie large amounts of friction and cost for reskilling workers. McKinsey highlights that automation is 14x more likely to affect low wage workers who are typically less protected by legal, technical and relationship moats. Bridgewater: Implications of Zero Cost Work. McKinsey: Future Jobs. Winner Takes All Dynamics.
How Fast? Gradually and then all at once. Azeem’s 2024 outlook explores how seemingly sudden previous technology transitions were. At the early stage this looks like a steady drip feed of automation-inspired layoffs, for instance Duolingo last week, to pick one of many examples. Azeem. Duolingo.
Image: Speed of transition from horses to cars. Via Exponential View.
Around the world countries continue to take different regulatory approaches.
China has released a sanctioned dataset for training language models. It’s something like 5% of the size of the data sets used for top performing Open Source models. So Chinese companies must choose between performance or political & legal risk if they use other data sets and their model says something it shouldn’t. Link.
The EU AI Act contains many provisions. A good summary I came across on LinkedIn highlights quite how many. Link.
For those involved in building Language Models the New York Times vs OpenAI case has got top billing and, if it goes to court rather than settling, will set industry-shaping precedent. Organisations we work with aren’t training from scratch, but some are fine tuning Open Source models. For them, the main question about training data is reputational, not legal. How would I feel if it leaked to customers that I had trained on that? NYT Case. OpenAI response.
Learning More
Newsletters
Scott Belsky for helping to rethink design and systems
Ethan Mollick for the questions that everyone is asking - his LinkedIn is good too!
Azeem Azhar’s ExpView is excellent across a range of forward-looking tech analysis and has a community, ExpDo, for paid members
A User Guide to public AI tools from Exponential View. Link.
Ranking of models for cost, speed and usefulness. Link.
The Lighter Side
A picture can say 1000 words, and some of those words might be ones you’d tried really hard to stop the model saying. Link.
A refreshing take on AI helping students with their homework. Link.
“In Japan, an AI system designed to distinguish croissants from bear claws has turned out to be capable of identifying cancer cells.” Link.