The type of crystal I keep with my deck depends on the deck itself. Sometimes I will pair them based on the deck's theme (I pair tigerseye with my Animal Spirit oracle for example and moss agate with my tarot of trees) but other times I will pair them based on the vibe of the deck or the feeling it gives me (I paired rose quartz with my Nature's Whispers Oracle because I felt a lot of love and comfort when using the deck, and paired rainbow obsidian with the Wild Unknown Tarot because the deck felt very grounding to me).
Right now I'm putting together a mix of stones for my copy of the Bonefire Tarot. I wanted something fiery both because of the deck's theme and because I am also a fire astrology sign. I feel connected to the deck that way. I am putting a tiny glass bottle with citrine (which is a stone that makes me feel more energized and also resonates with fire signs), a charm featuring my sign, onyx which is grounding and protective, bloodstone which is the gemstone assigned to my sign, and lepidolite which I've found helps with soothing and relieving stress. Sounds like a lot but they're all small little pieces that fit in the bottom of my bag and to me I feel like adding stones and charms to stay with my deck helps to make it more personalized and mine.
But if you don't believe in/or feel drawn to adding crystals, herbs, whatever with your cards, don't.
Everyone approaches the cards differently and no one faults anyone for how they go about it.