This might not be the right place to ask this question, but I know there are some very knowledgeable people who may be able to help.

If I were to outfit a canoe with a North Water spray deck
https://northwater.com/products/canoe-spray-deck
which puts mounting patches 5" below the gunwales drilled through the hull and a set of interior float bags that are independently attached to the canoe. How would you do this?

My thoughts would be to not use the attachment patches that come with the spray deck and instead thread a line 2.5" below gunwales running the entire length of the boat. This would allow for attachment of the spray deck on the exterior and bags on the interior. But I'm not sure how low I would have to go in order for the spray deck to be attached properly.

If I were to forgo that option would it compromise the hull if I had a lower 5" set of holes for the deck and higher 1" set for the bags?

Not really able to answer your question, but figured the hull and gunnel construction material might make a difference. What boat are you upgrading.
I had Dan Cooke at CCS fit a cover for my Bell Magic that uses snaps that are about a half inch below the gunnel. Nice and tight and not likely to leak nor come off in any situation where I was not also thrown out of the boat.