An Open Secret: British Open Source Intelligence during the Second World War


Ben Wheatley is a Honorary Research Fellow in the School of History, University of East Anglia and a former Teaching Fellow at the Defence Studies Department, King’s College London. His 2017 book, British Intelligence and Hitler’s Empire in the Soviet Union, 1941-1945 published by Bloomsbury Academic will be available in paperback from August 2018. You can follow him on Twitter at @DrBenWheatley.

Today we live in a networked world. Our reliance on the internet and social media has made open source intelligence (OSINT) a key resource for intelligence and security agencies across the world. However, the importance of OSINT is far from a modern development. The British government, for example, has placed the serious study of OSINT at the centre of its intelligence arsenal for the last 100 years (2018 being the centenary), but until now, to little fanfare.

Last year my book British Intelligence and Hitler’s Empire in the Soviet Union, 1941 – 1945 revealed for the first time the full extent of the role played by the Foreign Research and Press Service (FRPS) and its successor the Foreign Office Research Department (FORD) as an intelligence organisation during the Second World War.[1] The book highlighted that following the Nazi invasion of the Soviet Union in June 1941 OSINT (sometimes known as overt intelligence) obtained via the enemy and neutral press was the only regular source of intelligence available to the British government from the entire Nazi-occupied Soviet Union. This occurred as a result of the 1941 Foreign Office ban on all British covert intelligence and espionage activities (including SIS/MI6 and the Special Operations Executive) inside the Soviet Union and its sphere of influence in Eastern Europe.[2] In fact only one SOE operation was ever attempted in the Nazi-occupied Soviet Union. This operation (codenamed Blunderhead), which was initiated only following Soviet approval, ended in disaster as British agent Ronald Seth (who parachuted into Estonia in 1942) defected to the Germans and agreed to work for the Abwehrand later the SD.[3] Clearly then Churchill’s statement that SOE would ‘set Europe ablaze’ can in no way be attributed to its ‘operations’ in the Soviet sphere of influence.[4]

The British decision not to pursue covert intelligence operations against the Soviet Union was part of a change in attitude by the British towards their new Soviet allies that was designed to warm relations between the two states in wartime. It was thought that covert operations, if discovered, could lead to a possible collapse in the wartime alliance. As the alliance was of fundamental importance to Britain’s survival – the Red Army was, after all, engaging over three million German troops which might otherwise have menaced Britain’s position in Africa and the Middle East, or even the United Kingdom itself – the pursuit of covert intelligence against the Soviet Union was seen to be an unnecessary risk.

The Foreign Office ban on all direct work against the Soviets meant that any request for covert intelligence gathering inside the Soviet sphere of influence was dismissed by SIS.[5] This reluctance to endanger Anglo-Soviet relations was highlighted as early as July 1941 by the attitude of SIS to a proposal made by Admiral John Godfrey, the director of naval intelligence. Godfrey had written to SIS’ chief (or ‘C’) Stewart Menzies about the possibility of embedding intelligence officers in Moscow as he believed it was a ‘golden opportunity to obtain that intelligence regarding the USSR which we have lacked for so long’. ‘C’ believed this ‘would only result in compromise’ and argued that ‘the Anglo-Russian alliance against Germany’ had ‘not altered my policy of endeavouring to obtain USSR information’, but Godfrey must appreciate ‘that I have to tread very warily as regards any further steps I may take’.[6] The British ambassador and mission in Moscow, he continued, were ‘most anxious that nothing should be done which may in any way impede their efforts towards a successful collaboration with the USSR authorities, the result of which may have much influence in winning the war’. ‘C’ was clearly aware of the political ramifications of the proposal and this was a definite SIS refusal. In early 1942 the Foreign Office placed on record its doubt that any intelligence of sufficient value and importance could be secured in Moscow to warrant the risks being taken to secure it.[7] Further evidence of the total enforcement of this ban (and the rather understandable Soviet reaction to it) is provided by the notorious Soviet spy Kim Philby who recalled ‘MI6 was not engaged in any subversive and espionagework against the Soviet Union… MI6 was not permitted to engage in it then, the USSR was Great Britain’s ally. But Moscow didn’t believe it, it didn’t believe it for a long time’[8].

The origins of the FRPS lay in the First World War and the years 1918-1920, it was during this period that the historian Arnold Toynbee helped establish the Political Intelligence Department (PID). The PID was the closest the British government came to tapping academic knowledge for the war effort in the First World War.By 1938 war with Nazi Germany seemed inevitable to the civil servants in the Foreign Office who were acutely aware that there was a lack of intelligence gathering and propaganda capabilities. The Foreign Office staff was already fully extended in dealing with normal peace-time diplomatic activities and there was a real fear that the Foreign Office political intelligence services would become overwhelmed in wartime. Arnold Toynbee, by now director of the Royal Institute of International Affairs (RIIA), was contacted by Rex Leeper, head of the Foreign Office News Department, for assistance. Leeper had been a colleague of Toynbee’s during the First World War in the PID. Leeper and Toynbee agreed that, in the event of war, a large team of international affairs experts from the RIIA (aka Chatham House) and the universities would operate as an independent information and intelligence section for the Foreign Office. Toynbee’s academic group would supply the Foreign Office with a regular survey of the foreign press and would also prepare memoranda as required on the historical–political background of specific foreign problems. The new organization would be based at Balliol College, Oxford, and would assume the title Foreign Research and Press Service. Toynbee recruited eminent specialists in history and international affairs to his staff. The FRPS would soon employ 131 staff, eleven of which were professors whose salaries were paid by Oxford University, twenty-three worked as paid research assistants, twenty-four as volunteer researchers and seventy-four as assistants and clerical staff. Toynbee was driven by a strong belief that the study of history could prove useful in the formulation of rational policies for the British government to follow in wartime.[9]

Historians of the Second World War have erroneously assumed that due to censorship the Nazi and Nazi-occupied press contained nothing of value. In an October 1941 memorandum, the head of the Stockholm Press Reading Bureau (SPRB) Cecil Parrott clearly articulates the folly of this view:

With the greater part of Europe under enemy occupation, and many of the few surviving neutrals isolated from us (Britain) by belts of occupied territory, the problem of obtaining information from Germany and the countries she has occupied becomes more and more difficult to resolve. Hence the almost unprecedented significance of the European press as the only source today which provides regular information daily of conditions and developments in Germany and occupied territory.

It is often objected that the German and occupied press contains nothing of interest, as it is subjected to the strictest possible censorship and would not be exported if it contained anything of use for the enemy. Experience has proved the opposite. While revelations of great significance are seldom to be found, the sum total of material available, if scientifically read and treated, often provides a remarkably revealing and coherent picture of the conditions in occupied territories. Moreover the press of the few surviving neutral states, especially those whose frontiers are contiguous with Germany, often contains most useful indications of the trend of opinion in Germany itself.[10]

Such a statement would have been music to the ears of Arnold Toynbee and the FRPS’ staff. Indeed, Parrott’s memoranda, together with a glowing endorsement given to Eden by a February 1941 government enquiry into the role of the FRPS, would help to ensure that the FRPS became established as the British government’s main source of intelligence regarding the German-occupied Soviet Union (Foreign Office members were instructed to make full use of the FPRS’ ‘vast store of knowledge’).[11] In 1943 the FRPS absorbed the reconstituted PID, relocated to London and became an integral department of the Foreign Office. The new Foreign Office Research Department, as the FRPS was now known, retained Toynbee as its head. In Eden’s own words it had now become unthinkable that this ‘remarkable organisation’ should remain outside direct Foreign Office control.[12] Eden’s patronage of the FRPS had been vital to its survival, from an early stage the Secretary of State recognized the importance of having an organization that could read the enemy press and provide detailed reports for his office and the government at large. It was Eden who had requested the formation of the SPRB in September 1940, another sign of his regard for this type of intelligence gathering.[13] The FRPS could not have survived without the SPRB’s ability to regularly obtain and supply (via civilian air transport) intelligence from the enemy and neutral press.

One may ask the question why has the importance of OSINT in wartime gone so little commented on by historians?It is my belief that the answer lies in the appeal of covert intelligence and espionage operations. For example tales of the Bletchley Park code-breakers, the ‘Cambridge Five’ and secret missions behind enemy lines have blinded academics to the importance of the less glamorous but no less important role of OSINT. This remarkable oversight needs to be rectified or history runs the risk of giving a false narrative. However, the importance of OSINT during the Second World War is indeed beyond question. This is perhaps best illustrated by a January 1945 report written by the leaders of the Joint Intelligence Committee (JIC), which oversaw the direction of all of Britain’s wartime intelligence operations. The report declared that ‘even in war, much of the information which is of value to the Foreign Office and Defence Services is in no way secret’ and that the ‘foreign press’ was one of the ‘principal’ wartime intelligence channels. This was a significant statement that indicates the value and faith the JIC leadership had in OSINT during the Second World War. The authors of the report even declared that any proposed post-war Central Intelligence Bureau (similar to the USA’s then emerging CIA) should place OSINT at the forefront of its intelligence arsenal.[14] As Michael Herman (himself a former secretary of the JIC) has written: ‘the JIC spotted the significance of publicly available sources such as newspapers, fifty years before they became a fashionable subject… for 1944-45 this was indeed visionary’.[15] Further evidence of the vital role played by OSINT during the Second World War can be found in Sir Harry Hinsley’s (Bletchley Park code breaker and official post-war historian of British intelligence) observation that ‘of the total number of reports (by the Enemy Branch of the Ministry of Economic Warfare) some three-fifths were based upon the Press, broadcasts and official statements’.[16] Clearly then as the JIC, Herman and Hinsley indicate OSINT had an important role in the intelligence war in general, while my extant research has shown that in the Soviet sphere of influence OSINT was in the vanguard of the struggle against Nazism.

Although some German Enigma traffic was read from the Eastern Front by Bletchley Park this intelligence was largely of a military nature and often misleading. For example in 1943 the majority of decrypted intelligence was limited to the Luftwaffe Enigma ‘which was not very illuminating’.[17] In the West Enigma decrypts are often quoted as providing war winning information to the Soviets, the reality was quite different. Ultra intelligence had only a modest impact in the East, for example the Luftwaffe Enigma decrypts only made it clear that Operation Citadel (the German operational offensive around Kursk in July 1943) had begun some six days after the start of the battle, therefore British ‘intelligence gave no advance notice of the opening of the much delayed German offensive’.[18] For their part the Soviets preferred to rely on their own much more accurate intelligence streams such as photo reconnaissance, information from its agents (those besides Bletchley’s John Cairncross), its excellent razvedka-diversionary detachments and groups in the tactical and operational depths of the German defences.

OSINT on the other hand provided the only source of regular (daily) information of events behind the front in the civilian ruled areas. For those in Whitehall this was an invaluable source of intelligence which allowed them to formulate policy, particularly post-war planning, in the full knowledge of the current economic and social conditions prevalent in Eastern Europe. For example Anthony Eden described the April 1944 FORD’s paper on the Probable post-war Tendencies in Soviet Foreign Policy as ‘Excellent’, Churchill agreed to its conclusions and asked for it to be circulated to the Dominion Prime Ministers. This paper (which advocated cooperation over confrontation) formed the basis of the British government’s post-war policy towards the Soviet Union. Eden instructed the Post-hostilities Planning Sub-Committee (PHPS/c) to stick to the principles of the FORD paper in order ‘to foster and maintain the friendliest possible relations with the USSR’, this was something the rabidly anti-Soviet Chiefs of Staff were loath to do. In August 1944 Eden presented an adapted version of the FORD paper under the title ‘Soviet Policy in Europe’ to the War Cabinet and as a result the original FORD paper’s conclusions became the official stratagem of the British government when dealing with the Soviet Union in post-war Europe (a policy continued by Attlee’s Labour government from 1945).[19] As can be seen the FRPS/FORD work had real impact. OSINT also provided vital information which could be used in the formulation of post-war criminal trials, for example at various stages OSINT provided evidence of civilian slave labour, Jewish persecution and Nazi Germanization activities. OSINT in effect was a window into the brutal Nazi occupation of Eastern Europe.

As Sir Hinsley indicated, even away from the agreed Soviet sphere of influence in Eastern Europe and the Foreign Office ban, 60% of all economic intelligence in some government departments came via OSINT.[20] It is my belief that the coming years will see a major reorientation in our understanding of how the balance of intelligence was, and indeed still is, procured and utilised during wartime. As a consequence OSINT will be retrospectively viewed as forming a vital component in the intelligence war against fascism.

[1] Ben Wheatley, British Intelligence and Hitler’s Empire in the Soviet Union, 1941-1945(London: Bloomsbury 2017).

[2] Keith Jeffery, MI6: The History of the Secret Intelligence Service(London: Bloomsbury 2010) p.568, see also Dónal O’Sullivan, Dealing with the Devil, Anglo-Soviet Cooperation during the Second World War(New York: Peter Lang 2010) pp. 27-28. The Soviet sphere, by agreement with the NKVD, included: the Soviet Union (1941 borders), Finland, Romania and Bulgaria. The West’s sphere stretched from Norway to Spain and also included Greece. The Central European states were omitted from the September 1941 Anglo-Soviet Agreement.

[3] Ben Wheatley, ‘Revisited, The Security Services Investigation of British Abwehr/SD Agent Ronald Sydney Seth, Journal of Intelligence History, pp. 116-149, Vol. 16, No. 2 (2017).

[4] Louise Atherton, SOE Operations in Eastern Europe(Richmond: TNA 1995) pp. 23-27 see also her SOE Operations in Scandinavia (Richmond: TNA 1994), p. 10 and John Fisher, SOE Operations in the Balkans(Richmond: TNA 1998), pp. 14-18 and pp. 28-32.

[5]Jeffery, MI6: The History of the Secret Intelligence Service, p. 554.

[6] Ibid., p. 562.

[7] Bradley Smith. Sharing Secrets with Stalin, (Kansas: Kansas State University Press 1996), p. 114.

[8] Stephen Dorril, MI6: Fifty Years of Special Operations(London: Forth Estate 2000), p.8.

[9].R.H. Keyserlingk, “Arnold Toynbee’s Foreign Research and Press Service, 1939-43 and Its Post-war Plans for South-east Europe”, Journal of Contemporary History, Vol. 21, No. 4, (1986): pp. 541-44.

[10] TNA FO 898/252, Stockholm Press Reading Bureau: Intelligence Reports 1940–1942, Parrott Memorandum, 9 October 1941, p.1.

[11]TNA CAB 117/77, Studies on Reconstruction by the Foreign Research and Press Service of the Royal Institute of International Affairs 1941, Letter from R. A. Butler at the FO to Arthur Greenwood Chairman of the Cabinet Committee for Reconstruction Problems, 11 February 1941.

[12] Robert Longmire and Kenneth Walker, Herald of a Noisy World – Interpreting the News of All Nations: The Research and Analysis Department of the Foreign and Commonwealth Office – A History, (London: FCO 1995), p. 16.

[13].TNA FO 898/252, Stockholm Press Reading Bureau: Intelligence Reports 1940–1942, Letter from Mallet at the British Legation Stockholm to State Secretary Anthony Eden, 16 October 1941, p.1.

[14] Michael Herman. “The Post-War Organization of Intelligence: The January 1945 Report to the Joint Intelligence Committee on ‘The Intelligence Machine’.” In Learning from the Secret Past: Cases in British Intelligence History, edited by Robert Dover and Michael Goodman, (Washington: Georgetown University Press 2011), pp.35-42. .

[15] Ibid., p.21.

[16] Frank Hinsley, British Intelligence in the Second World War: Vol. 2(London: HMSO 1981), pp. 130-31.

[17]David Glantz, Soviet Military Intelligence in War (New York: Cass 1990), pp. 220-21.

[18] Ibid, p.221.

[19] Ben Wheatley, British Intelligence and Hitler’s Empire in the Soviet Union, 1941-1945 (London: Bloomsbury 2017), pp. 183-91.

[20] Frank Hinsley, British Intelligence in the Second World War: Vol. 2, (London: HMSO 1981), pp.130-31.

Image: Chatham House’s wartime Committee for Reconstruction meeting in the Institute’s Common Room, 1943, via Wikimedia commons

2 thoughts on “An Open Secret: British Open Source Intelligence during the Second World War

  1. This essay provides an excellent description of the value of OSINT, explains why it has been misunderstood and dismissed by historians, etc. and also explains how and why ULTRA fell short in critical ways. But it would be much stronger, and easier for lay readers to understand, if examples were provided of just what OSINT told whom, and when, and what the original sources were.


Leave a Reply to Jonathan Beard Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s