Colonial papers and the ugly legacy of empire

Just because it is easy with hindsight to condemn or mock the excesses of the time doesn't mean our parents and grandparents weren't right to be nervous
British troops round up Kenyan locals for interrogation during Mau May uprising
British troops round up Kenyan locals for interrogation during Mau May uprising. Photograph: Popperfoto/Popperfoto/Getty Images

What links the suspected murder of China-based British businessman, Neil Heywood, with the part-publication of long-hidden colonial files from the last decade of the British empire? Why, Foreign Office and M16 secrecy, of course. Throw in Wednesday morning's news that Jack Straw is being sued by a Libyan over his alleged role in extraordinary rendition and the link is irresistible.

When I bumped into Labour's Denis MacShane on Tuesday, the MP was confident that he would seriously embarrass William Hague over the Heywood affair when he challenged him later that afternoon over the slow way in which the alleged crime emerged. You can catch it at column 171 in Hansard – though the foreign secretary seems to have survived unscathed.

Hague is reportedly asking the Beijing embassy, its spooks and diplomats, why it didn't flag up the gossip about light drinker Heywood's suspicious death and rapid cremation – the cause attributed to alcoholic excess – rather sooner. Straw, who last year denied in public that the then-Labour government had connived in any form of rendition, must be asking the FCO why he is being accused of having signed something which suggests otherwise.

In other words, the spirit of Yes Minister's Sir Humphrey lives on.

Of course, Foreign Office officials and the espionage community across the Thames at Vauxhall Cross are going to try to avoid making waves with mighty China over the death of a businessman operating in the shadows with powerful local friends. It's easy to imagine officials wanting to hush up the troublesome murder of Russian exile Alexander Litvenenko if he hadn't been poisoned so conspicuously with polonium in a London hotel. Hush things up and burn the files – or risk costly lawsuits, not all of them noble ones? It's not hard to follow their train of thought.

So what should we make of those surviving archives which are finally being released from the government's high-security centre in Buckinghamshire to the clutches of historians eager to get hold of them in the National Archive, close to Kew Gardens on the Thames in west London?

Tony Badger, Master of Clare College, Cambridge, who is overseeing the transfer, says that most of the colonial records were weeded and burned, that the rest should have been published under the old 30-year rule in the 80s and that the whole affair is "embarrassing, scandalous".

And so it is. It may be mere gossip, no less enjoyable for that, that British and US officials – ie the state department and FCO – fretted that Barack Obama Snr was just the type of Kenyan scholarship student to "fall into the wrong hands" in America and emerge "anti-American and anti-white". Since his own dad, Onyanga Obama, was detained and possibly tortured during the Mau Mau emergency, that's hardly surprising – as I have written before. President Obama seems content to overlook the offence; he has more important things to worry about.

But the mistreatment of the 1,500 citizens of Diego Garcia, exiled in the 1960s to create the huge Indian Ocean military base so crucial to American global power projection, cannot be but a poignant story, heartlessly cruel and probably avoidable. Again, the initial injustice and expediency has been compounded by secrecy and obstruction.

Kenya, Cyprus, the Malayan emergency in which British colonial forces inflicted a rare defeat on a communist-led jungle insurgency – the files which have survived will still provide plenty of useful insights. Malaya fascinated the Americans, who tried and failed to repeat its counter-guerilla techniques (remember "secure hamlets"?) in Vietnam a decade later.

Yet context usually provides some mitigation for such actions, occasionally justification, which it is wise not to dismiss out of hand. Nothing reported in Wednesday's newspapers about British excesses would at first glance match what the Belgians did for decades in the Congo – Europe's worst colonial scandal, the Germans in what is now Namibia or the French in Algeria, the legacy of whose imperial retreat lives on.

That's no excuse, but it is context. So is the atmosphere of the 1950s which I dimly remember, when Stalin's armies occupied the eastern half of Europe and word was slowly emerging of the scale of atrocity committed during and after the Terror and the second world war – 5 million Stalin victims in addition to the 20 million Russian dead at the hands of the Germans.

With China in Mao's hands we were all scared of what the "red menace" might be about to do to the rest of us. Paranoia was particularly acute in the United States of the McCarthy era, when bad things were done in the name of liberty, but often mixed up with race. Ed Pilkington reported only this week on the 40-year solitary confinement of two black Americans caught up in the tail-end of that madness and still protesting their innocence.

But just because it is easy with hindsight to condemn or mock the excesses of the time doesn't mean our parents and grandparents weren't right to be nervous – or even that the often-brutal suppression of the Malayan insurgency didn't do modern Malaysia, conspicuously prosperous and relatively open, a historic favour. Just think about neighbouring Burma.

The cold war was often frightening and always corrosive, but it did have a point. Whether its peaceful ending in the collapse of the Soviet Union and its bloc after 1989 has ushered in better times or merely a relatively benign interlude remains to be seen. Edward Lucas's new book Deception (Bloomsbury, £20) suggests we are foolish to underestimate the continuing threat to Britain posed by Russian espionage – often conducted not by goons and oligarchs, but by modern young Russians with PhDs and nice accents. US worries about cyberwarfare focus on China, as Nick Hopkins reports.

As Julian Assange's WikiLeaks exercise showed, though many readers were too indignant to notice, the cable traffic it revealed tended to expose more of the double-talk of the states Washington has to deal with than outright US perfidy. But the condemnation of Britain's failings by Harvard professor and author of Britain's Gulag (Holt/Cape), Caroline Elkins, does warrant a gentle riposte.

Her history of Britain's suppression of the Kenyan Mau Mau revolt – very brutal too in its own way – won a Pulitzer prize, though Oxford's David Anderson covered much of the same bloody ground in his Histories of the Hanged: Britain's Dirty War in Kenya and the End of Empire (Weidenfeld and Nicholson) in the same year of publication.

Most of this ugly stuff is known, it's often the context which is forgotten. Poison gas tests in an isolated corner of Botswana? They never actually happened, but Britain feared the Germans might use it first – as they did in 1915.

But Americans should always resist the easy temptation to take too much moral high ground over the Brits. They have their Kenyas and their Diego Garcias too and are usually as selectively amnesiac as we are. Thus, the predominantly black, southern city of Washington DC boasts a haunting Holocaust Museum. But when I looked at the bench on which Jews were not allowed to sit, I thought of the countless benches in the American south on which – in the same decade – African Americans were not allowed to sit either.

As for Native Americans, when I lived in the US I was often struck by how small were the newspaper archives discussing their continuing plight and problems, compared with those of African Americans. Some might see the whole country as their Diego Garcia.

So almost the last thing I did before returning to Britain was visit the Pine Ridge reservation – home of the Wounded Knee Massacre of 1890. It is not a pretty sight. History is moving on and the west is slowly being judged for its failings as well as its achievements.

From here on others will be writing their own accounts of the past 500 years of European expansionism, now over. Some of them, I dare say, will be committing atrocities of their own against the weak too.