Some on AI in Education

[copied from https://x.com/Dirk_Englund/status/1745328474754084986?s=20 as requested by ppl who stopped twitter]

Why I've been particularly behind on emails: A serious re-addiction to coding, thanks to @openai. At the time of #ChatGPT release, I was working w/ Dr Donnie Keathley on the new course notes for MIT EECS' brand-new "6.2410 Quantum Engineering Platforms"...

It was shocking how well this works -- try for yourself: [https://barosandu.github.io/intro.html] .. thanks to the architecture built on python simulations + markdown + context-awareness + generation of figures from python (helps our LLM), the 'hallucination rate' could be kept very low... but not zero. (related: [https://community.aws/posts/genai-for-edu]) For example, consider this quantum optics problem:

Consider an optical mode initialized in Fock-state superposition (|1>+|3>)/sqrt(2). This mode propagates through a channel with power transmission 90%. What is the probability of measuring n=1 on a photon number resolving detector, to 2 significant figures? below .45 36.4% .45 27.3% .46 18.2% above .47 18.2% 22 votes · 1 day left

Spoiler-alert: GPT4 and all other LLMs I've tested have been getting questions like this confidently wrong: [https://chat.openai.com/share/6da8c413-64f4-4d65-9a5c-5ec13463492f]

To get around this, over the summer, I started trying to involve iterative coding. The reason: if we interpret the nat'l language problem 'q' into a mathematically precise input 'x' to a physics solver (eg, qutip for example), then we can solve the answer through computation. A beautiful fact about physics and math is that if you're given input x, you can predict the experiment output (distributions) through a deterministic map x--> f(x). So together with the other things, this iterative coding allowed me to reduce the hallucination rate (the confidently wrong answer rate) to nearly zero, conditional on giving an answer at all (autogpt & open-interpreter are awesome, but they would get lost in my ad-hoc ranked lists or knowledge graphs of quantum optics problem solving). Also, iterative coding tends to burn through @openai credits like crazy. (More on that in a later post: some exciting developments to discuss)

Fortunately, during the summer of 2023 I had gotten a couple of great CS undergraduate coders interested in the project, starting with @HankStennes and Hunter Kemeny and eventually we had a really awesome team of volunteer developers on GitHub, leading to upgrades ([https://chattutor.org/static/index.html]) and our eventual open-source release of ChatTutor ([https://github.com/ChatTutor/chattutor]) back in October.

A few days ago, my co-teachers @prajit_dhara Dhara and Dr Chuck Choi were able to complete a brand-new course for the NSF Center for Quantum Networks (@CQN_ERC ) Winter School: [https://docs.google.com/presentation/d/12USlmI1TyJERcgK5Wnq2W3RLTFZeD28BwBGNznhnlSo/edit#slide=id.g1bf1da94ade_0_0] + [https://docs.google.com/presentation/d/12USlmI1TyJERcgK5Wnq2W3RLTFZeD28BwBGNznhnlSo/edit#slide=id.g1bf1da94ade_0_0], deploying ChatTutor as a personal co-pilot to 50+ participants, thanks to some heroic last-minute assistance by Adrian Ariton and Alexandru Ariton.

I'm very thankful to students for great questions.. and so far no complaints of physics hallucinations by any of the 4 teaching staff :) ! (I won't count that ChatTutor used slightly unconventional notation for a Bell state )

The leaps in GenerativeAI in the last years have been breathtaking. I've felt compelled to work as much as I could to try methods to base genAI in truth in the fields I know about in engineering, physics, and engineering, and other fields where the truth is knowable/computable..

Along the way, I got dreadfully behind on email. I'm happy to meet with people in person or to talk over phone or zoom. Faced with tons of emails per day, I've found it impossible to keep up while still doing science. So, for now, sorry to folks I owe responses.

Still waiting for that useful AI assistant.

For now, it appears that keeping up with email is harder than automating quantum physics.