re-enterant scenes!wowie this was a trickyboi to figure out, but I think I managed to get a decent-working solution. The algorithm consists of 2 parts: 1) detecting scene transitions, and 2) figuring out if the new scene is a duplicate. Part 1) was solved using perceptual hashing, and watching the delta of the frame hash between frames. When it spikes, odds are that's a scene change. Part 2) was a bit trickier, but boils down to hashing every frame and associating it with a scene + scroll values. That way, when a scene transition is detected, I can check the hash-maps to see if the "new" frame is actually one I've seen before. There is a bit of extra work that gets the technique working with elaborate fade-in animations / animated backgrounds, but it's not magic or anything. What's left? Well, 1) needs to be improved, since the current delta threshold value i'm using has been pulled out of thin air, and while it gives reasonable results on many games, ideally, it should be more robust 2) is pretty good though! All that I need to do now to be super happy with it is serialize the scene data and enable loading / saving it to disk! Once that's done, the initial vision of being able to automatically map out a game will have come true! exciting!