left_adjoint
/
miscellaneous-docs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

final draft

This commit is contained in:
clarissa 2023-06-13 15:34:04 -07:00
parent 00dc26cadb
commit 5664169d61
1 changed files with 18 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
newsletter7.org
View File

@ -80,8 +80,24 @@ But in the weeks since it launched in beta testing, the voice-based, AI-powered
"The AI was not programmed to do this and has seemed to go rogue," Marjorie told Insider. "My team and I are working around the clock to prevent this from happening again."
#+end_quote
And okay so I want to be careful when I say this next part. I think it's fair to call building the "virtual flirty girlfriend" a kind of sex work and I bring this up not to say "well of course people would try and do this" but rather the opposite. Sex work always has to have really well-defined boundaries of what is and isn't allowed, what services are and are not being provided. She has boundaries for what people are allowed to do with her voice and image, boundaries she is allowed to have. But people are using the inherent fuzziness of large language models to violate those boundaries. And I'm not sure if there's going to be any real way around this kind of problem for a true LLM.
And okay so I want to be careful when I say this next part. I've seen some people have the reaction to this story of something like "well of course people did that". And I don't think that's an appropriate reaction, at least not if your analysis stops there. You can be unsurprised that people would violate her boundaries because unfortunately there are a lot of misogynists out there. But we have to acknowledge that even if what she's selling is a "flirtly fun girlfriend" version of herself she has boundaries for what people are allowed to do with her voice and image, boundaries she is allowed to have and that should be respected. But people are using the inherent fuzziness of large language models to violate those boundaries. And I'm not sure if there's going to be any real way around this kind of problem for a true LLM.
The reason why you can't easily give an LLM guardrails is the flipside of why you can use an LLM for all these different tasks that we've never trained it for: it is capable of responding to text prompts that reflect the myriad ways you can concretize an idea into words.
So how on Earth do you put guardrails around that? People can very easily
So how on Earth do you put restrictions on that? I've linked to it before but Simon Willinson talked about a related problem here: https://simonwillison.net/2022/Sep/16/prompt-injection-solutions/
I wish I had answers but as Willison quotes in his blog post, the problem with prompt injection is that it isn't an error it's the language model doing what it's supposed to do.
So this might seem like an awkward segue but this kinda leads into another piece that I think is worth thinking about: Against LLM Maximalism (https://explosion.ai/blog/against-llm-maximalism)
This is an essay by someone who is very experienced in the scene of natural language processing (NLP) talking about how he thinks things like LLMs can be part of making an application but can't be the whole of the application, that they need to be modules within the larger structure.
Now, his examples are more about using LLMs for data analysis and things like that, not to make chatbots, but I feel like the fundamental idea stil applies. We need to be willing to apply older more deterministic tricks for natural language processing and natural language generation with the LLM as only part of the larger thing, rather than treating the LLM like a big black box of an application where you feed it text and spit back out its response uncritically.
This is kinda interesting to me because it has a parallel to how we've been talking about LLMs in higher ed: yes you can use them to create drafts, generate ideas, analyze text but it can't ever be the final word. You need to check the behavior of the LLM, check what it does, always treat it as part (i.e.module) of the workflow (i.e. application).
Again, I don't really have answers for what this should even look like. I don't think any of us do. I think we're all trying to wrap our heads around what it means to use this hyper-general models that can do almost anything you can imagine with language and not at all predictably.
And on that note that brings us to the last thing I want to link to which is that the Department of Education released this document called Artificial Intelligence and the Future of Teaching and Learning (https://www2.ed.gov/documents/ai-report/ai-report.pdf)
I think this is absolutely something worth reading and I'll have a lot more comments about it next week but I think I'll keep this piece to under 2500 words for once!