A very good question Sunstallion
Your astrologers are right on both counts BUT (and you knew there would be a BUT):
It all depends on what you mean by Compatibility
You are assuming (as I did initially) that they meant, what we normally think of as compatibility - a two way or mutual process. That is you get on with them and they get on with you. For this to be done you need both natal charts and you need accurate times of birth (for these purposes THEIR chart can be out by an hour or possibly up to two hours)
However there is a weaker form of 'compatibility' - are they someone that you would get on with (though the reverse may not apply or apply as strongly). In this weak form of compatibility, you are really only interested in where their planets lie in YOUR chart.
If you know their accurate time of birth you can look at your chart with their planets, Ascendant and MC placed in your houses, which not only will give you some indication of compatibility in general but also WHERE you are compatible, e.g. as friends, lovers, work colleagues, siblings, parent and child, etc. (and obviously you can reverse the process for them, and look at your chart in terms of their houses)
As their natal data becomes less accurate, you will lose some of this data. If their chart is more than a couple of hours out, their Ascendant will be difficult to establish, even through approximation. Even when slow, the Moon moves at around 1 degree every 2 hours, so again, only knowing the date, rather than the time can put the Moon about 6 degrees out, if you take it's midday position.
Knowing the date only means that you can look at all the planets, but the Moon with reasonable accuracy relative to your chart. But the Moon is important, especially with compatibility because it says something about empathy. Also the Ascendant is a key factor, saying something not only about your physical body but also a lot about your temperament (the Moon is an almost equal contributor, the Sun making up the difference). Thus with no accurate Moon and no accurate Ascendant you can't do any real assessment about their temperament (and that is important to you) and obviously nothing about your compatibility with their temperament.
So the best you can hope for is a very general indication, if all you know is their date of birth. It might look promising if their Venus is in your Seventh, but if You are Phlegmatic/Sanguine and the are Choleric/Melancholic, their mood swings will drive you nuts. But with no accurate birth time, you can't do an accurate assessment to detect this possible incompatibility.
Finally, if you are approximating the Moon's position by taking it's midday placement don't forget to allow for any difference between local time when you were born and UT time, given in the ephemeris for the current Moon position. This becomes especially important if they are international friends or happen to have been born out of the country. If they were born in Australia and you were born in California, their date of birth will be 'wrong' - in your terms they could well have been born the day after the date of birth given, because Australian time is well ahead of USA time.