My new AI-powered research experiment

I wish I could be prouder of my new ten-part writing project. It’s a somewhat pulpy 50,000-word novel set in Shanghai in the early 20th century.

Problem is, I am not the writer. That’s John, the guy in the image below.

(John can also be found at Substack as #tdsexpert. Pronounced “TDS Expert,” just to be clear.)

After 40 years, 20 books, and 200 academic papers, John is going to clean out his office and retire next year. But not before his novel is completed. The novel tackles the life of Satomi Hajime, Japan’s “Opium King,” acquitted Class A war criminal, and 1916 graduate of the Tōa Dōbun Shoin. The Tōa Dōbun Shoin (literally East Asia Common Culture Academy) was an international language and trade school run by the Japanese in Shanghai from 1901 to 1945, and John knows everything about it. Meanwhile, his knowledge of international trade in China in the early twentieth century is encyclopedic.

Smart as John is, there are a couple things he does not know. First, all those books in his office are not going to become a named collection in his university library, because the library is not going to have space for any of them. Nor will any of the other East Asian libraries spread around the country. Actually, used bookstores are not going to want them either. Their back rooms are already stacked to the ceiling with boxes of books waiting to be processed. Yes, John’s books are bound for the landfill.

John would also be surprised at what his office looks like after he finally vacates next spring. Yes, a large desk will be tucked in back by the window overlooking the leafy campus.? But on that desk will be a keyboard and three monitors and no books or papers. More astonishing to John, a long table is pushed up against the wall where his bookshelves used to be; a huge blue recycle bin on wheels sits at one end. On the table is a diabolical machine with a three-foot handle, a machine that can only be described as a guillotine. The new occupant reaches into a box of old books she bought from a nearby university library, which has been liquidating any books not signed out for at least 20 years. She slides one up against the guide in the guillotine. Krrchrr-CRUNCH, the spine on the book falls away neatly, and the new office occupant scrapes it into a waiting garbage can below. She gives the 600 loose pages a pat and slides them into the scanner sitting next to the guillotine. Sixty seconds later she scoops up the loose pages and dumps them in the recycle bin, reaching for a new book in the box.

Okay, I will not insult your intelligence by stringing this along further. This essay explores how AI-enhanced digitalization of the research process is already starting to replace some aspects of John. And while this technology may not necessarily diminish the value of John’s expertise and his books, it changes the significance of his expertise in fundamental ways. To make my case, I ask ChatGPT to write a historical novel on a topic that has fascinated me for almost 30 years: a much-maligned, much-misunderstood business school run by the Japanese in prewar Shanghai. To generate the novel, I have created a GPT in ChatGPT called “Ahen-O,” literally “Opium King” in Japanese.

First, some background. I first encountered the Tōa Dōbun Shoin in a journal article at the University of Toronto in 1996, soon after completing a stint studying business Chinese in Beijing’s University of International Business and Economics. While the history books judged the school harshly, dismissing it as some amalgam of spy training academy and imperialist planning apparatus, I was immediately interested. As I scanned the school’s 1901 international business curriculum, it occurred to me that was the education I wanted—and did not have—when I stepped into the fevered entrepreneurial environment of Beijing in 1994. In early January, I am bound for Aichi University, Tōa Dōbun Shoin’s postwar successor institution, to work on some of these projects. I have been thinking lots about AI, too: AI is about to fundamentally disrupt research projects with large, complicated archives like the one I am tackling.

AI has plenty to disrupt. Academic research is specialized, jargon-ridden, dry, and deliberately inaccessible. Cynical estimates suggest fewer than ten people thoroughly read an academic paper about a topic like the Tōa Dōbun Shoin. Most peer-reviewed papers never even get cited by other scholars, a key indicator that the article is having limited impact. The newest cutting-edge research is hidden behind high paywalls. Interested in my recent paper about the business schools Japan ran in China at the end of the 19th century? You can pick it up here from the publisher’s website for $53 USD.

John’s novel about the Opium King challenges this whole system. My point here is not that AI can now produce something comparable to what the snobs in Sweden choose every December. The novel is quite bad—and potentially inaccurate in some places—but its scope far exceeds anything I could manage on my own. (The opium den dream is inspired by Coleridge's "Kubla Khan" poem.) Thus, I borrow the fiction format to pose some simple questions. What aspects of the research process—finding, organizing, and writing—can AI already do better than humans? How are these expansive capabilities going to change the way academic research and universities are organized?

In short, AI-generated content is synthetic, creative, and accessible in all the ways traditional academic research is not.

AI can synthesize.

After 40 years, the knowledge of talented researchers like John on a specialized topic and its adjacent fields becomes profound. Problem is, John’s knowledge cannot be categorized, summarized, queried, or even saved. When John walks out of his office for the last time, his wisdom is lost to his students, his books never to be opened again. Once his 20-page CV disappears from the faculty website, we even lose the thread linking his research into a cohesive body of work.

In contrast, consider AI’s astonishing synthetic powers. Japanese, English, and Chinese documents that make up the “Ahen-O” GPT’s knowledge files are diverse. They include not only documents related to Satomi and the Tōa Dōbun Shoin’s development, but also content about Chinese comprador middlemen, local Chinese banks, Japanese involvement in the opium trade, and treaty port life in Shanghai. One file is even a scan of the third book in the Chinese language textbook series published by the Tōa Dōbun Shoin in 1925, which discusses China’s cotton, tea, sugar, tobacco, and wool industries.

Implications of AI’s synthesizing capabilities are far-reaching. In my hard drive, I have several thousand pages like the one you see below, a page from student records of the elaborate research trips into China’s hinterland performed by Tōa Dōbun Shoin students in their third year.

As we might expect from travel logs, the content in these pages varies from ridiculous to profound. Thus, these documents should be skimmed, not read, since their value is not uniform. Unfortunately, the slightly archaic Japanese in combination with some grainy, barely-legible text makes this text a laborious read rather than a casual skim, even for a native Japanese speaker. Yet scanning software like FineReader in combination with AI performs this task easily. (It will ignore what it can’t decipher.)

In the process, the digitalization process brings the journals back to life. For example, the above page is one of 285 other pages chronicling the 1915 excursion the Opium King took in the third year of his program at the Tōa Dōbun Shoin. One of several cohorts, his team took a long, perilous trip through Beijing, Xi’an, and Wuhan, much of it on foot. AI allows us to search for magical moments like the following: “We arrived at our destination, a small village, and the owner of the inn sent someone to buy meat for us. After waiting for about an hour, they returned with salted beef, which had a dark brown color resembling dried persimmons. We had hoped for fresh meat, but since it was night, they told us it was unavailable. We tried cooking the meat with various ingredients—dried bonito, miso, and pickled fish—but when we tasted it, the smell was so bad that we could not eat it.”

AI can create.

The first paragraph of our novel goes like this. “The steamer belched black smoke as it inched into the bustling port of Shanghai, the boat’s rust-streaked hull groaning as it gently rolled in the brown, oil-streaked waters by the wharf. Satomi Hajime, clutching a weathered leather satchel to his chest, leaned over the railing, eyes wide with wonder. The air was thick with the mingling scents of brine, coal, and food frying in stalls near the dock.”

Surely this bit of crap-tastic, AI-generated fiction can’t have anything to do with serious academic research, you say to yourself. True—maybe. Academic research is very poor at imagining things, even things that might open up some new, productive avenue of research. For instance, past English-language research about the Tōa Dōbun Shoin has not even bothered to imagine how a 17-year-old kid like Satomi arriving in Shanghai from Fukuoka experienced the place, and how the intensely stimulating environment of the treaty ports shaped the school’s development. Satomi’s gradual transformation into Japan’s Opium King tells an entirely different and equally important story from that told in the history books. A little digging into the school’s graduates reveals a whole culture of maverick, entrepreneurial, independent thinkers, some of them ethically challenged like Satomi. That slow entry of the boat into the harbour is important.

AI makes knowledge accessible.

I finally settled on John’s AI-generated likeness because I liked his stare. The stare says to me, “Ask me anything. Go ahead, challenge me. I can answer anything related to Shanghai.” AI has given us access to John’s brain, and invites us to ask any question, even a dumb one.

Certainly, the “Ahen-O” GPT is at best an experiment, a work in progress. I have configured the GPT specifically to lay out a popular novel, and it does not always do that or other tasks well. The GPT is fuzzy, vague, and occasionally misleading. But compared to formidable rows of books in Aichi University’s archives, which was all we had five years ago, the GPT is amazingly accessible. Most important, this accessibility has been conferred on the non-specialist. A tinkering kid could ask the GPT for a story and be equally entertained, enlightened, or informed.

I close with one point about language accessibility. In some respects, academic research about some esoteric topic like life in Shanghai in the early 20th century has been sharply constrained by language. Famous China specialists like John Fairbank spent their entire careers plumbing the depths of Qing China documents written in bureaucratic language (the vestiges of which are particularly prominent in Taiwan and Hong Kong bureaucracies today.) Few others could understand or process it. Academic research revolved around the painstaking standardization and consistent use of foreign language terms and concepts. That challenge is going to gradually fall away.

Please check out the first chapter, or have a go with my Ahen-O GPT.

Based on readers’ suggestions, I want to add and swap out instructions / knowledge files for the Ahen-O GPT, follow new directions for the plot, and gradually make the historical picture the GPT creates more accurate and realistic.

?