San Jose Wants Its Workforce to Make Their Own AI Assistants

San Jose believes artificial intelligence can save its employees valuable time — and it plans to teach them to make use of the technology. A 10-week training program is getting employees into the game.

“I’d heard the name of ChatGPT being thrown around, but never truly explored it until this course,” says Data Analyst Stephen Liang, who was among the first cohort of city staff to start training in September 2024.

That training involved weekly one-hour sessions that familiarized staff with the technology and had them design their own department-specific AI assistants — that is, AI tools designed to perform specific tasks to assist in an employee’s role. Employees can create assistants to help them more quickly find information or analyze large amounts of data, for example.

“These assistants range from anything that you can really do with data to text analysis to conversations and interpretations of images,” says Albert Gehami, the city’s digital privacy officer.

The tools have saved valuable time on a range of data processing tasks, city staff and officials say. This is encouraging the city to push further. Now — with about 50 students having completed the training — San Jose plans to expand the offering to about 15 percent of the workforce by the end of next year.

During his training, Liang created a tool to help identify recurring patterns in 311 requests, so the city could understand which problems needed more attention and resources. Residents filing 311 requests online can select a preset category or select “other” and type their concern into a text box. The 311 team might receive tens of thousands of typed requests, and manually reviewing them all to identify themes was a time-consuming task, Liang says.

“We have a conservative estimate of [this taking] about 500 hours a year,” Liang says. “Somebody, that person being me, would have to sit there and sift through every single row of data to try to even gather or comprehend what residents are reporting to us.”

Now the AI assistant does this task. In its analysis, the tool recently found that residents had recurring concerns around water issues and garbage service.

“The real impact goes beyond the time saved for me as a data analyst,” Liang says. “It translates to more time [spent] on areas where we're able to explore the more complicated problems.”

Another program participant created a tool analyzing parking records, to assess whether people are able to find parking around the airport or if they’re left driving between different lots hunting for a spot, Gehami says. Another employee created a tool to process purchasing receipts.

The training aimed to reach people who were familiar with their own departments and needs, but who hadn’t experimented much with AI. Having departments themselves, not IT, make the tools was important, too: “We might know the technology, but we don't know all of the intricacies of everyone's work. … Once they know a little bit about the tools, they come up with the clever, brilliant ways to use them that actually saves them time and provides better services.”

Limits and safeguards

Of course, there’s a limit to the tools. Not all use cases students tried worked. Liang originally wanted to make a chatbot that would analyze all 311 requests coming in — not just those typed into the text box.

“But the problem with that was, I saw that [the data] wasn't granular enough for the custom GPT to be able to give me results that I needed to better understand what residents were reporting,” he says. “I had to narrow down my scope into one specific category.”

Students in the course learned that ChatGPT has historically performed better at reading PDFs than Microsoft Word documents, Gehami says. And the employee who created a receipt-processing tool discovered they could only have it process a certain number of pages before ChatGPT would start fabricating nonexistent purchases. AI tools make mistakes, and they have a tendency to convincingly pass off falsehoods as real. Gehami advises thinking of the tool as “an overactive intern”— it’ll work hard and fast but its results shouldn’t be trusted without verification.

“It’s a constant learning of the limitations,” Gehami says. Sometimes, staff will report that using ChatGPT shortened a task by one hour, but that they think they could trim an additional three hours if the technology was more capable. "In my mind, it gets me excited for the future … as the technology continues to evolve, we are ready to take advantage of it.”

Even with the need to spend time carefully fact-checking, AI can still be a time-saver when used in the right way. The tool is helpful for searching through vast amounts of documents or data to find answers that would take a human many hours to hunt down. If the AI assistant is made to cite its source, it’s then fairly quick for an employee to check whether the answers are actually right, Gehami notes. For example, the tool can sift through the city’s thousands of pages of environmental codes for ones that might apply to a frozen waffle factory seeking a new permit. An employee can look up the codes the tool identified to check if they’re relevant.


The city’s AI assistants also need to preserve residents’ trust by keeping their information private. That means making sure information entered into the assistant doesn't become part of a private vendor’s training data and potentially disclosed elsewhere, when ChatGPT answers some future question. The city selected an AI product that lets officials opt out of having their data used for training and makes this setting the default.

An AI-Forward City

Nearly all San Jose departments have had employees participate in the training. But given the sheer size of the city workforce, not everyone will have the opportunity. However, staff can view recordings of the training online, and several hundred employees have participated in a one-off intro class explaining the tool and the associated privacy and accuracy concerns. Gehami also expects those who have participated in the 10-week training will share their knowledge with others. Already, several teams have reached out to Liang for advice on building their own custom GPT models.

These AI assistants are far from a first step for San Jose. It already uses AI for purposes like language translation and bus route optimizations. The city issued guidelines on employee use of generative AI a few years ago. And it leads the GovAI Coalition, a collaboration among governments of all levels, in which members share with each other best practices, use cases, standards for government AI policies and vendor contract language and the like.

Liang says he’d encourage other employees to keep an open mind about using AI. “When the calculator was invented, it didn't replace the accounting. It just made their workflow a little easier.”