@themanual4am
Not sure what you mean by "deduplication." That could mean many things. There's sensory integration, which in AI might just be the "hidden state" of the network once it's been trained and given some observables. There's also the problem of "identity," where an intelligence groups together observations and understands them to be "the same object." This comes up all the time in robotics, for recognizing / tracking objects in a video, or for self-monitoring in motion control. I guess one more "duduplication" phenomenon I can think of is categorization. Intelligent systems generalize across many experiences to make a stereotype about a category, such that any new instance of that category evokes all the features of the stereotype. This is a form of memory and an example of compressing sensory data. AI does this by vectorizing input tokens, and then clustering those vectors, or with auto-encoders that "fill in" noisy / partial data.
Is this getting at your question, or am I off track?
2/2