My own sense of an "esoteric" aspect to the TDM, or at least a representation of a tradition or teaching or special narrative, which had a particular purpose beyond gaming, is that it is there. For me, the two extremes of projecting something foreign on it as being its *true* significance and stridently denying there is any such significance or purpose are merely somewhat hardened and competing agendas to make the cards into what one wants them to be.
To understand that the aristocratic decks often portrayed the aristocrats who commissioned the cards, or that some of the other images were common in the "triumph parades" or that the Mamluk cards portrayed basic polo sticks etc is helpful, imo, in grounding them in their historical context.
But as LRichard mentioned, to assume we know everything about how people viewed these images and symbols, (whether in a parade, a mamluk court, in bedtimes stories, in the streets, brothels and gambling dens, craft guilds or whatever) is misguided.
As well, while the Visconti-type decks appear to have notable variations in terms of additions, deletions and cards we dont even know whether they had or not, and the mamluk and Chinese cards, as far as I know didnt even have an elaborate trump system, what seems clear to me is that at least in the Marseille tradition, a careful (up to a point) systematization and ordering occurred and was preserved. As well, as Wilfried Houdouin shows in his wonderful deck and book, there seems to be convincing evidence that a pretty damned refined (sacred) geometric grid was being applied to layout and proportions. In this regard, at the very least, in the TDM cards we are probably looking at the kind of knowledge about structure and proportion (ie Vitruvius etc), generally kept within the domain of architects, masons, "initiated" craftspeople, great artists etc.