If the file manager doesn’t currently have the sole focus then I do not really see why remapping Cmd-Arrows would impact anything at all when the preview window has focus.
I’m somewhat confused by this.
When previewing in Finder the Quick Look window normally has the active focus the entire time, and there is seldom a reason to go to any effort to switch the focus back to the Finder window. This is precisely because the QL window in macOS is transparent to both the arrow keys alone, for jumping around between different files in a single folder, and to the Cmd+Arrow shortcuts, so you can navigate around to different folders without closing the preview window. I would not want the preview window to interfere with folder navigation just because it has the focus. The way QL works in this manner has always been very handy.
You’re talking about changing context, but in practical terms I can’t really tell exactly what you mean by that where the preview window is concerned. I’d have to experience your implementation of what you’re trying to do. The only thing I can think of is (from your example of the web browser) that it would make sense to grab the arrow keys, and possibly the Cmd+Arrow shortcuts, when actively selecting text inside the preview window. (Which is actually not possible in macOS. It acts more like an image browser, even with text documents.) But outside of that context the arrow keys and Cmd+Arrow shortcuts should otherwise be forwarded to the file manager window for navigating files and folders.
If that kind of context is all you mean by this then I’m confused for no good reason. As usual. But it seems like we may be thinking about the basic function of the preview window differently. As if it’s more like the Preview.app application in macOS, and meant to open with the space bar like QL but act somewhat independently of the file manager window. That’s not the way I think of a “preview” plugin, so having the preview window take over the Cmd+Arrow keys wouldn’t normally make sense to me. Outside of the active text field context.