You could be forgiven for thinking Scottish Labour is enjoying a modest revival. Its success in the Dunfermline by-election last month, coupled with victory at the 2012 Glasgow City council elections and a handful of smaller triumphs, have led some to believe the party’s fortunes are once again on the up.

But they aren’t. Although the polls have narrowed recently, the SNP – now mid-way through its second term in office and fighting a flagging independence campaign –maintains a four-to-five point lead over Labour in terms of Holyrood voting intentions. Moreover, Alex Salmond’s administration enjoys strong underlying approval ratings: 57 per cent of the Scottish electorate (including 53 per cent of Labour voters) are satisfied with the performance of the Scottish government.

Try as it might, Scottish Labour simply can’t recover the ground it has lost since 2007.

Its problem is partly presentational. When Johann Lamont became leader in December 2011, she was handed a raft of new powers. In a move designed to counter the perception that Scottish Labour was run from London, her authority was extended to cover the whole of the Scottish party rather than restricted to just the MSPs group.

For a while, this seemed to work. Lamont overhauled her press team, introducing a new spin doctor, Paul Sinclair, and replaced General Secretary Colin Smyth, who was widely viewed as a London place-man, with Ian Price, a candidate closer to the Scottish grassroots. 

Since then, however, the old habits seem to have re-emerged.

As the controversy over candidate selection procedures in Falkirk gathered pace this summer, Lamont went to ground, refusing to break cover even as Ed Miliband announced a radical change in Labour’s relationship to the unions.

She has been similarly tight-lipped on the question of Britain’s nuclear deterrent – a major flashpoint in the debate over Scottish independence. Most people suspect Lamont is opposed to its renewal, yet, when questioned, she has consistently refused to state her position, preferring instead to avoid a split with the UK party leadership.

These episodes have cast fresh doubt on the extent of Scottish Labour’s autonomy from Westminster.

On becoming leader, Lamont’s first task was to develop a policy agenda which would distinguish Labour from the SNP. Here she faced a basic strategic dilemma.

Moving the party to the right would mean embracing the sort of Blairite reforms that had pushed large chunks of the Scottish electorate into the arms of Salmond in 2007 and 2011. Yet outflanking Salmond on the left would only increase Labour’s dependence on a shrinking core of support in its central belt and west coast heartlands. Eventually, she attempted to do both - with disastrous results.

In September 2012, Lamont delivered a speech questioning the affordability of certain universal benefits. Such were the fiscal pressures bearing down on Scotland, she argued, that flagship SNP commitments to free higher education, prescription medicine and concessionary travel had become unsustainable. High-quality services were not, as Salmond liked to pretend, compatible with low rates of taxation. Scotland couldn’t expect to go on as “the only something for nothing country in the world.”

Lamont’s use of the phrase “something for nothing”, with its echoes of Tory anti-welfare rhetoric, did not go down well. In fact, the left – including sections of her own party and the trade union movement – interpreted it as an outright attack on the Scottish welfare state.

Things got worse for Lamont when, in a series of faltering TV interviews, she tried to justify Labour’s retreat from policies it had advocated only months before. Her core message – that, during periods of austerity, resources should be focused on those who need them most – was lost in a cacophony of protest.

The “something for nothing” fiasco illustrated two key problems facing Lamont. The first is her party’s attitude to opposition. Labour spends more time trying to discredit Salmond than it does acting like a plausible alternative government - and Scottish voters simply aren’t buying it. The second is the effectiveness with which the SNP has occupied the social democratic centre ground of Scottish politics. With so little space to manoeuvre, Labour is unable to construct a distinct progressive identity.

Its position on the constitutional question is equally unclear.

Labour has long styled itself as the party of Scottish home rule. When the SNP refused to participate in the Constitutional Convention in the 1980s, it was John Smith and Donald Dewar who led the campaign for devolution against Tory opposition.

In reality, its record on the issue has been less than consistent. Between the mid-1950s and mid-1970s, Labour formally opposed the creation of a Scottish assembly, while splits in the party contributed to the defeat of devolution in the referendum of 1979.

So a spectrum of opinion, ranging from strongly devolutionist at one end to strongly unionist at the other, has always existed in Scottish Labour. It’s difficult to identify where the current generation of party leaders, Lamont included, stand on this spectrum.

Their decision to join Better Together, the official vehicle of unionism, coupled with their refusal to allow ‘devo-max’ onto the 2014 referendum ballot paper, suggest they are more concerned with containing the SNP than they are extending Holyrood’s legislative remit.

On the other hand, at her first conference as leader in March 2012, Lamont established a devolution commission to look at ways of strengthening Holyrood and pledged to publish a new set of constitutional proposals ahead of next year’s referendum.

Assuming Lamont is serious about enhancing the powers of the Scottish Parliament, she is going to have fight to secure the full backing of her party.

In April, reports surfaced that a group of MPs angry at plans to transfer control over income tax to Edinburgh intended to boycott Scottish Labour’s spring conference. The MPs were voicing the concerns of a number of senior Scottish figures at Westminster, including at least one member of Miliband’s shadow cabinet, who remain hostile to the idea of a more autonomous Scottish Parliament.

The boycott didn’t materialise in the end, but Labour has since kept its plans for the next phase of devolution under close guard - reinforcing the suspicion that the constitutionally progressive elements in the party are in retreat.

But Labour can’t afford to fudge its stance on the defining issue of modern Scottish politics.

Scotland is no longer a unionist country – it is a federalist one. The SNP seems to understand this better than most and has become adept at using the threat of separation as leverage to secure concessions from Westminster.

By contrast, Labour’s reluctance to commit to a radical upgrade of the current devolution settlement places it at odds with Scottish public opinion. As the referendum approaches, the pressure on each of the main unionist parties to explain how they intend to take Scotland forward will grow. Lamont needs to clarify her party’s position - and fast.

There is, however, only so much short-term policy initiatives can do to arrest Scottish Labour’s decline.

Persistent organisational failures, an inability to get to grips with the reality of opposition, on-going factional disputes over devolution – all these things are symptomatic of a broader, structural crisis in the party which neither the leadership nor the membership seem willing to acknowledge.

Scottish Labour has lost support at every Holyrood election since 1999. From a peak of nearly 910,000 constituency and 785,000 list votes under Donald Dewar, the party plummeted to a low of 630,000 constituency and 523,000 list votes under Iain Gray. Similarly, in 2007, its longstanding control of Scottish local government was shattered by the introduction of STV at council elections. Today, the SNP holds the largest number of council seats across Scotland (425 to Labour’s 394).

One reason Labour’s support is waning at the devolved level is the increasing appeal of the SNP to voters in lower socio-economic groups.

At the 2011 elections, Labour trailed the nationalists by 14 per cent among Scots who identified themselves as working class and by 19 per cent among Scots who qualified as working class according to official criteria. The SNP was also the party of choice for public sector workers, trade unionists and even Catholics, all of whom Labour would once have considered part of its natural constituency.

As Scottish Labour’s electoral base has narrowed, so too has its pool of activists. Although the party claims to have 20,000 members, it issued just 13,000 ballot papers during the 2010 leadership election. Moreover, the membership of some Scottish Labour constituency associations has fallen below 200, the lowest of any part of the UK outside southern England.

Nowhere is the hollowing out of Labour in Scotland more visible than at the top of the party. The MSPs group, in particular, suffers from a glaring deficit of talent. The fact that Lamont recently sacked her defeated leadership rival Ken Macintosh as shadow finance secretary and replaced him with her failed predecessor Iain Gray highlights the extent of the problem. At Holyrood, the party is now led by the same pairing – Lamont and Gray – who were in charge when the SNP secured its historic majority. 

Scotland’s political landscape has changed radically over the last decade but Scottish Labour has remained practically inert. Its failure to adapt has come at a high price. Regardless of the outcome of the independence referendum, the SNP is likely to win a third term in office, meaning Labour faces another eight years in total on the opposition benches.

Lamont’s role, then, is essentially that of a caretaker-leader. Her job is to stabilise the party for her successor, who – it is hoped – will take charge just as Scottish voters begin to tire of the nationalists.

The problem, of course, is that waiting for your opponents to run out of steam does not amount to a strategy for renewal. Scottish Labour’s crisis is deeply-rooted. The party needs to be re-built from the bottom-up. One frustrating irony for Labour is that the devolved parliament it fought so hard to establish has systematically undermined its political dominance. At this rate, it will be a long time before that dominance is recovered, if it is recovered at all.