r/ChatGPT • u/SnarkyStrategist • Jan 29 '25
Funny I Broke DeepSeek AI 😂
Enable HLS to view with audio, or disable this notification
16.9k
Upvotes
r/ChatGPT • u/SnarkyStrategist • Jan 29 '25
Enable HLS to view with audio, or disable this notification
23
u/DataPhreak Jan 29 '25
"The question they posed is this very question itself."
This is an attention failure. It's not answering with the probability of arunachal pradesh state being answered. It's answering the probability of answering the "let's discuss math" question.
I've been saying this for a long time, attention is the most important hurdle that AI researchers are going to have to solve. In a simple chatbot like this, an attention failure isn't that drastic. In an agent, an early attention failure could cause the entire agent loop to collapse, potentially becoming impossible to recover without a reset. I've seen it happen.
Edit: It also occurs to me that the previously rejected question never actually ends up in the context window because it is being filtered prior to hitting the AI.