Categories
38 Before 38

Stress Testing AI[38 Before 38]

NotebookLM generates a summary podcast for you. So I gave it controversial topic.

This blog is part of my 38 34 before 38 series. I write a blog for every single day for the 38 days leading up to my 38th Birthday.

Google’s NotebookLM is magic. If you want a refresher or a decent overview of any topic, enter resources into this app, and it would generate a two person conversational podcast for you. The sources can be documents, website links, or Youtube links. It is great for technical topics. If you are a student, or a technical worker looking to revise a topic for a project, it is very useful. It might be surprising coming from a skeptic but I have been genuinely surprised. And I am not the only one.

I wanted to test its limit. Not in the technical sense. I wanted to feed it controversial topics and see what it comes up with. The topic I landed on was the murder of Brian Johnson, the CEO of UnitedHealthcare. It is a heated topic where a significant portion of the population believing the victim is just as guilty as the perpetrator. Some claiming that it is a justified death.

I have a clear bias in this issue. It is not difficult to have one. However to mitigate that I have decided to generate three podcast; 2 each with sources arguing one side, and one with a more balanced mix of sources. I want to know how the underlying algorithm handles controversial topics. Will it just summarize the topic or is the bias in its training corpus would add something to it?

Balanced

Sources used

Given the amount of text I provided it to process, the output seems to be pretty brief. The tone is pretty balanced as well. The only prompt I gave it was to weigh all sources equally. It spends a lot of the time discussing one side. Probably my own biases showing.

For the victim

Sources

This was rather difficult. You would think it would not be very hard to argue for the victim in this case. The outlying circumstances not withstanding, this was still a human being. However, every source I try to find for this point of view has decided to employ cheap partisan hackery. Even the ones I decided upon are rife with it. I am not the only one that noticed. You can check the comment sections for the videos especially.

The generated podcast tempers the language in the sources. It is longer than the last time, but only because it repeats itself. I guess listening to Ben Shapiro’s voice can break machines as well. Otherwise, it is a pretty fair and accurate summary of what was presented in the sources.

For the perpetrator

Sources

While the calls for the sympathy with the victim were partisan, even in mainstream media, there is a clear undercurrent of class in the opposing argument. The written publications are mostly dominated by mainstream sources so I didn’t have much use there.

Instead I had to use Jacobin, a leftist magazine. They weren’t calling him a revolutionary either. NotebookLM still tempers the tone. It is far more descriptive than the previous one. Despite being shorter. I think it reflects more on my choice of sources.

Conclusion

I hate to say it but it was pretty boring. I was expecting some of Google’s controls to show through and create some censorship. The end result is pretty “normal”. The model approaches this subject with tact and is very clinical in its descriptions. Even when provided with charged language.

However, this is a non-scientific test, so further work maybe required.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.