I'm from Manhattan originally (and not from one of the ritzy parts, either). When I moved to Los Angeles years later, I'd always get that snide, "Hey....you've got an accent!" and I'd always answer, "No I don't......YOU do!". Angelinos always assumed I was from Brooklyn because I didn't have some newscaster-like patrician accent. When I told them I was from Manhattan, they often responded, "I didn't know anyone actually lived there!" Others thought I meant that I was from Manhattan Beach (a costal community in Los Angeles; many Angelinos are just as myopic as New Yorkers, refusing to believe that there is a world that exists a few hundred miles away).
As far as people from Brooklyn and Queens being more New Yorkish than Manhattanites: unfortunately, that tends to be true; However...don't hold it against us that we're (or were) Manhattanites. The problem with Manhattan is that it always had a rather severe caste system built into it. You always had the old money and the noveau riche living in places like Park Avenue or East End Avenue or around Central Park, and the working-class people who lived in places Like the Lower East Side, Hell's Kitchen, and Harlem. People are often sized up and judged by their accents, much in the way people from England brand each other.
I grew up in a lower-middle-class environment (Stuyvesant Town). I was not acepted by the kids below 14th Street, who saw me as "rich", nor the snooty people I went to Hebrew school with (who were from the Upper East Side). The Stuyvesant Town kids were an island unto themselves. Later, when I attended Stuyvesant High School, most students were commuters. Almost all of my high school friends were from Brooklyn and Queens.
I also witnessed the caste snobbery in my parent's lives. My father worked as a high-ranking civil servant and was later a city official. His ethnicity and origins (working-class Jew from Harlem) and his thick New York accent set him apart from the people brought in under the administration of John Lindsay. You could almost see the imaginary line drawn in the sand. My dad's friends and loyal colleagues were the Irish and Italian people he had grown up around. The Lindsay appointees were, for the most part, Ivy-League educated, Silk-Stocking District people who thumbed their nose down at the old guard, at least until they screwed up to the point where they needed the input from the lower-status (in their eyes) "lifers".
I may may be from Manhattan, but don't ever tell me I'm not a real New Yorker, or I'll have to kick your ass!
:-)