We used to classify ourselves as 'upper', 'middle' and 'working' class. Today, there are the super-rich, the damned well-off, those lucky to have a job and managing to survive, plus a largely ignored underclass written off as hopeless and shiftless. It is that underclass's children we should worry about.

The Communist Manifesto of 1848 said: "The history of all hitherto existing society is the history of class struggles." The difference now is that there is no struggle; the well-off are safe in exclusive enclaves with two BMWs in the drive (personal number-plates, of course) and their two-and-a-half children at posh private schools. Meanwhile, the needy and neglected have neither the will, the energy nor the power to do anything about their plight.

Labour's deputy leader and Equalities Minister Harriet Harman did not quite signal a return to the Old Labour barricades in what had been billed as an attack on the class system at the TUC conference. She toned down her prepared speech, did not mention the word 'class' and dropped a key passage saying social class, rather than gender, race, sexual orientation or disability was the main reason why people failed to reach their full potential: "What overarches all of these is where you live, your family background, your wealth and social class."

Perhaps Harperson knew she would be pilloried as the privately-educated relative of a countess, who controversially chose private education for her own sons. Yet none of that prevents her from talking sense about the growing gap between rich and poor and the unfair advantages of the privileged.

Harman announced a national equality panel to help the Government tackle the "scars of inequality". Right-wingers and elitists will condemn this as an attempt to level down British society, when it is really about making available to all the good things of life like a liveable home in a decent neighbourhood and access to higher education.

Labour has ground to make up. The Mass Observation project which monitored diarist's feelings during the Second World War recorded one worker's hope for change: "There'll have to be more equalness. Things not fair now. Nobody can tell me they are. There's them with more money what they can ever use. This ain't right and it's got to be put right."

In the years that followed millions of people from the 'lower class', like myself, got apprenticeships, company training or higher education. We became upwardly mobile and, yes, affluent. Now these ladders have been largely removed and, while "them with more money" got still more, an inequality gap was created. Gordon Brown, in his present 'mea culpa' mood, admitted last week Labour had not done enough to improve social mobility, despite a decade-long drive to combat poverty.

Harman agreed: "By the time they reach the age of six, a less able child from a wealthy family will have overtaken a more able child from a poor family." Graduation rates for students from the richest fifth of society have doubled to nearly 50%, while the number of graduates from the poorest families has crept from 6% to 9%.

The Tories scoffed at Harman as being "stuck in the class warfare rhetoric of 20 years ago" but their rhetoric is worse, even offensive. When Gordon Brown pointed out that Old Etonian David Cameron had little in common with ordinary electors, another former pupil of Toffs' Academy, sneered: "He is attempting to re-open a class divide that long ago disappeared. He and his party are refusing to admit the existence of the real divide in our society. The bottom 20% of society – the group that supplies us with the chavs, the losers, the burglars, the drug addicts and the 70,000 people who are lost in our prisons and learning nothing except how to become more effective criminals."

For good measure, he added: "The super-rich will always be with us. They are mainly harmless." Admittedly, this was the buffoonish Boris Johnson, but he is presently the most powerful Tory in office and betrays the ingrained contempt of many Conservatives for 'the lower classes'. 'Chavs' is a derisive dig and there is no attempt to understand why people are 'losers' in the life-race or why so many turn to crime and drugs. In the same way, David Cameron descended on one of the most deprived areas of Scotland to talk about our 'broken society' but only offers moralising piffle about changing taxes and benefits in favour of marriage.

Deeply disturbing is the superior attitude that underpins much right-wing thinking. This is the belief that there will always be people worse off than others not because of unfairness but because it is in their natures to be underlings: "Some people run faster than others, some people think better… this is the way civilisations become strong." In other words, it is a good thing that the rich get richer and the poor poorer. That is not class, but the product of opportunism and naked greed.

There are those who cannot run fast or think better. And there are those who, through apathy or laziness, are happy to live on state support in sink estates. Realists may write these off as a lost generation but, unless we are going to create a dangerously divided society, we must rescue their children from the same hopeless fate.

Real class cannot be bought; it is the way you conduct yourself, particularly towards those less fortunate. A government with real class will ignore snob rhetoric and keep fighting that old fight.