What are the impacts of urban agriculture programs on food security in low and middle-income countries: a systematic review

Abstract

Background

Urban Agriculture is considered to contribute to improved food security among the income poor in urban contexts across developing countries. Much literature exists on the topic assuming a positive relationship. The aim of this review was to collect and analyse available evidence on the impact of urban agriculture in low and middle-income countries.

Methods

We employed systematic review methods to identify all relevant and reliable research on UA’s impact on food security and nutrition. Only impact evaluations that set out to measure the effectiveness of UA interventions on food security, as compared to the effects of not engaging in UA, qualified for inclusion. Studies had to have a comparison group and at least two data points.

Results

Systematic searches resulted in 8142 hits, and screening of abstracts resulted in 198 full texts identified. No studies met the review’s inclusion criteria. Therefore, the review found no available evidence that supports or refutes the suggestion that urban agriculture positively impacts on individual or household food security in low and middle-income countries. The largest proportion of studies at full text stage was excluded based on study design, as they were not impact evaluations, i.e. they did not have a comparison group and at least data points. Two observations were made: Firstly, searches yielded a range of studies that consider associations between UA and certain aspects of food security. Secondly, there is a large pool of cross-sectional studies on UA’s potential to contribute to increased food security, particularly from west and east Africa.

Conclusions

The research currently available does not allow for any conclusions to be made on whether or not urban agriculture initiatives contribute to food security. The fact that impact evaluations are absent from the current evidence-base calls for increased efforts to measure the impact of urban agriculture on food security in low and middle-income countries through rigorous impact evaluations. With regard to systematic review methodology, this review alludes to the value of compiling a systematic map prior to engaging in a full systematic review.

Background

The emergence of urban agriculture

The twenty-first century has often been described as ‘the first urban century’. Unprecedented rural–urban migration has led to rapid urban growth. Whilst in 1900 a mere 13 per cent of the world’s population lived in urban areas, the UN-Habitat [1] estimates that by 2030 this level will have risen to 60 per cent. Furthermore, virtually all of this population growth over the next few decades will be absorbed by cities in low and middle-income countries and thus increase pressure on often already exhausted urban resources and administrations.

Among the most pressing needs of any urban agglomeration is to achieve food security. Urban populations depend on reliable and stable availability of food products as well as affordable and convenient access to them. High levels of urban income poverty paired with rising food prices, however, often make the formal urban food supply system unaffordable to the urban poor. An informal supply system, consisting of street vendors, informal markets, home-based enterprises as well as Urban Agriculture (UA), since exists alongside formal interventions. These informal networks predominantly satisfy the urban poor’s demand for cheap and easily accessible foodstuffs.

Approaches to urban agriculture

Whilst urban decision makers and academics alike have identified UA as the most beneficial and promising pillar of informal food supply systems [2–7], the evidence for such claims is unclear. Although UA has been an integral part of urban livelihoods^a throughout human history the concept came only to the fore in the late 1980s/early 1990s, evoking interest among international donors and development practitioners [6]. A United Nations Development Program (UNDP) report compiled by Smit and colleagues in 1996 [7] estimated UA to be reaching some 800 million urban dwellers who used UA as a livelihood strategy in the early 1990s. A number of studies with promising titles such as ‘hunger-proof cities’, ‘Agropolis’ and ‘Cities feeding people’ [8] indicate the potential generally ascribed to UA. Critics of UA quickly pointed to the weak empirical evidence of some of these studies and the low overall scale of UA amongst urban poor [9]. During the first years of the urban century, UA had therefore slipped past the focus of the international development community. Yet the peak of global food prices in 2008 shed new light on the idea of locally produced food products and household subsistence production. Urban agriculture subsequently once more was considered as a major intervention to improve urban food security [10].

For example, the City of Johannesburg in its Growth and Development Strategy (GDS) 2040, identifies UA as its main intervention to address food security within the city [11]. On a global scale, the UN High Level Task Force on the Global Food Crisis [12] identified UA as an important strategy to alleviate urban food insecurity and to build cities that are more resilient to crises. A joint FAO/World Bank paper of the same year [13] also expressed that “the World Bank and FAO (…) will promote [UA] related programs and projects in the context of the MDGs and more specifically MDG1 ‘Eradicate extreme poverty and hunger’ and MDG7 ‘Ensure environmental sustainability’.” The FAO published the ‘Urban Producers Resource Book’ [14] as an outcome of its ‘Food for the city’ programme. FAO’s ‘Food for the cities’ program forms part of a wider network of organisations, consisting of the UNDP/UN-Habitat ‘Sustainable Cities Program’, the IDRC’s ‘Urban Poverty and Environment program’, and the Resource Center on Urban Agriculture and Food Security (RUAF), which all strongly advocate UA as a tool to address risks associated with urban food insecurity. In the light of this strong support for UA one must note, however, that some fundamental questions regarding UA remain unanswered. Renewed interest in the topic did not necessarily converge with new knowledge about UA. Little is known about the true extent and impact of UA in urban livelihoods.

Definitions of urban agriculture

Urban agriculture is a complex concept. A large variety of urban farming systems exists internationally, with varying characteristics depending on local socio-economic, geographic and political conditions, further complicating a universally applicable definition. Luc Mougeot developed the currently most widely used definition of UA in 2001 [15]. Using technical criteria of UA he explained that, ‘urban agriculture is an industry located within (intra-urban) or on the fringe (peri-urban) of a town, a city or a metropolis, which grows and raises, processes and distributes a diversity of food and non-food products, (re-)using largely human and material resources, products and services found in and around that urban area, and in turn supplying human and material resources, products and services largely to that urban area’ [15]. For the purpose of our review, this translates into an understanding of UA as a social intervention, geographically constrained to urban and/or peri-urban areas, involving any form of agriculture with the aim of improving the food security of actors involved.

A wide variety of produce results from UA and can best be classified according to their respective methods of production. Horticulture, animal husbandry, aqua culture and forestry can all be found in urban locations and generate products ranging from, inter alia, fruits and vegetables, dairy products, meat, fish, herbs and fire wood. In terms of end-points, UA’s products can either be used for consumption, surplus sale or trade and commercial activities. Actors in UA display a similar diversity. Whilst early literature assumed UA to be a livelihood strategy almost exclusively used by low-income groups and recent migrants to the city with the aim to increase household levels of food security, most scholars today identify a wider range of actors in UA [16]. The popularity of roof top gardens in many high-income countries for example shows the increased reach of UA.

Urban agriculture and food security

Urban Agriculture is thought to increase food security through two main pathways: improved access to food, and increased income [6]. This relates to a broad understanding of food security, follwing the 2009 Declaration of the World Summit on Food Security, which defines the state of ‘food security to exist when all people, at all times, have physical and economic access to sufficient, safe, nutritious food to meet their dietary needs and food preferences for an active life’ [17].

We will briefly outline these two presumed paths. The first of these assumes that home-grown foodstuffs increase the total amount of food available to a household and thus can prevent hunger and malnutrition. At the same time the availability of fresh, home-grown food products, in particular fruits and vegetables, advances the nutritional status of household members and thereby impacts positively on health outcomes. Direct access to food allows households to consume a more diverse diet that is richer in valuable micronutrients. Especially animal husbandry is believed to provide an important source of animal protein, which is commonly limited in poor households’ diets. Studies on UA and its impact on nutrition focus on dietary diversity and kilocalorie consumption [18]. In his 1998 analysis of child nutrition and UA in Kampala, Maxwell [5] also connected the aspect of maternal care to UA, arguing that mothers who engaged in UA, as opposed to other forms of non-farm employment away from home, have an increased ability to take care of their children. This was in turn considered to positively impact levels of child nutrition. However, even proponents of UA highlight the fact that there is currently no detailed empirical evidence for UA’s impact on nutrition levels [18, 19].

Secondly, UA is considered to increase household cash income. Domestic producers can either save income, as the household limits its need to purchase food, and/or increase income by selling or trading their products. Higher cash income at household level, in return is considered to be positively linked with food security, as households are believed to have greater access to food products both in terms of quantity and quality. This relationship however depends to a large extent on the calorie elasticity of income, ie the relation between a change in income and a change in calorie consumption [20]. Households with a low elasticity might not experience improved levels of nutrition linked to the increase in income when income is not spent on more nutritious food. Given the low input costs of UA, most scholars nevertheless believe it to have high potential in addressing urban poverty and food insecurity [21]. Figure 1 below depicts these two pathways.

Figure 1

Urban Agriculture’s two pathways to increased food security.

Critics of urban agriculture

Luc Mougeot [15] in 2001 has observed whilst “little could be found in the academic literature which would condemn UA at large … opposition has tended to come … from urban planning, public health and environmental circles.” The absence of easily accessible empirical data on UA’s scale and impact may explain urban planners’ reluctance to embrace the concept. Jac Smit, an outspoken proponent of UA, estimates that about 800 million people are involved in UA worldwide. His estimations came as a result of a number of visits to major urban centers in developing countries, funded by UNDP in the early 1990s, and a range of national surveys. Accurate data on the scale of UA is further limited by different survey designs and differences in conceptual definitions. While scholars [22, 23] once identified Lusaka as the urban agriculture capital, a recent baseline study by the Africa Food Security Urban Network [8] found that only three per cent of households in Lusaka currently use UA as a livelihood strategy. The same problem arises with regard to assumed impact of UA on urban livelihoods. Ellis and Sumberg [9] point to the absence of control groups in research regarding UA and criticise that “(UA) claims too much by equating all food production in towns with improved food security for poor people”.

Urban agriculture and the need for a systematic review

Hence, the need for a systematic approach to gather and synthesise available data on the impact of urban agriculture is evident. Urban planners and decision makers cannot be expected to base their policy recommendation on what Zezza and Tasciotti call “anecdotal evidence” [18]. Systematic review methodology provides means of identifying, synthesising and analysing the findings of a range of rigorous studies to answer a focused question. Petrosino and colleagues describe systematic reviews as “the most reliable and comprehensive statement about what works” [24].

A number of existing systematic reviews have touched on the issue of UA and food security. Berti et al. [25] reviewed the effectiveness of agriculture interventions in improving nutrition outcomes. Even though they did not make specific reference to the urban sector, Berti and colleagues’ findings might indicate trends applicable to urban households. The review of nutritional outcomes of 30 agricultural interventions found that the majority of interventions increase food production, but that this did not present a direct link to improved nutrition. Of the interventions that fostered levels of households’ nutritional status, most used a multiple approach focusing on nutrition education, amongst others, in combination with increased food production. Home gardening was found to be the most successful agricultural intervention in increasing levels of household nutrition. This finding is likely to resonate with UA interventions, as home gardening presents a common feature in various types of UA. Limitations in this review include the fact that conclusions are made which are based on studies with very high risk of bias.

Masset et al. [26] conducted a similar systematic review in 2011, which focused on agriculture interventions and levels of nutrition for children under the age of five. This review was registered with the EPPI-Centre at the University of London. Again, this review observed that agricultural interventions did increase levels of food production. However, this was not positively linked to improved levels of child nutrition either. The review further found that household diets do change on account of the presence of agricultural interventions, but that this change was not related to improved child nutrition as no improvement in levels of micronutrients consumed by the participants could be identified. The review’s conclusions are limited by the fact that the report does not aggregate its analysis by the level of included studies’ risk of bias, despite the acknowledgement that included studies are heterogeneous in terms of rigour of study design.

Lastly, a systematic review by the Dutch Ministry of Foreign Affairs’ Policy and Operations Evaluation Department [27] focused on the question which agriculture interventions improve food security. It identified increased production, improved value chains, market regulations and safe legal tenure as the four main pillars. Although the review claims that it considers urban food security, it neither differentiates between rural and urban food security impacts in its synthesis, nor does it account for such distinction in the reviewed impact studies. According to the review, improved irrigation and the use of genetic crop modification have been found to increase agricultural production in significant ways. Value chain interventions benefited farmers through improved income from the sale of cash crop. Market deregulations had an ambivalent impact on food security. Whilst the reduction of monopolies and the lowering of government involvement were believed to increase food security during crisis situations, its long-term impact differed greatly amongst the studies. Finally, improved land tenure benefited food security in all reviewed cases. This finding might be important in the context of UA, as it supports proponents’ calls to give urban farmers access to land and change discriminative urban policies to legalise and support agriculture interventions in urban areas. Since only a minority of urban farmers engages in UA for commercial reasons, interventions that focus on value chains and market regulations have limited impact on UA. The usage of genetically modified crops and better irrigation techniques might also be seen as an attribute of large scale farming projects. Individual and household farmers which present the majority of urban farmers and practice agriculture on mainly small plots within the city might not be able to make use the above techniques to increase their agricultural output.

As a result, the impact of UA on food security and nutrition is currently unanswered. Much literature has been published on the topic assuming a positive relationship, and the concept enjoys the outspoken support of international development agencies such as the IDRC and UNDP. Most research, however, lacks empirical evidence and few studies have generated reliable facts about the scale and impact of UA. Bearing in mind UA’s potential to make a meaningful contribution to securing food security among urban populations, this systematic review was developed in response to perceived gaps in the current evidence-base.

Objectives of this review

The aim of this review was to collect and analyse available evidence on the impact of urban agriculture in low and middle-income countries. We sought to provide a solid evidence-base for policy-makers, practitioners and members of the international donor community on the feasibility, benefits and cost of urban food cultivation. Specifically, this review attempted to answer the question: What are the impacts of urban agriculture programs on food security in low and middle-income countries? In addressing this question, we focussed on outcomes that measure levels of food security at the individual, household and/or community levels.

According to the definition adopted at the 1996 World Food Summit in Rome “food security exists when all people, at all times, have physical and economic access to sufficient, safe and nutritious food to meet their dietary needs and food preferences for an active and healthy life” [28]. This reflects a broad approach to food security, including the elements of food availability, food accessibility, food reliability, food quality and food preference [8]. Food security is most commonly defined in terms of the three pillars of: availability (including consistency of that availability), access (with the specification of not just access to food, but access to sufficient food for a nutritious diet), and use [13]. The FAO add a fourth pillar, that of stability and apply it to all three of the others i.e. the stability of availability, of access and of use [29].

Levels of food security and nutrition are interdependent with households’ socio-economic status. The economic impact of the usage of UA as an income-generating scheme in order to purchase more or different food is therefore also considered in this review. Sales of domestically produced food stuffs can be measured either in terms of quantity or with regard to their monetary value.

Methods

We employed systematic review methods as promoted by the Collaboration for Environmental Evidence. In doing so, we sought all the relevant and reliable research on UA and its impact on food security and nutrition, in order to synthesise it in meaningful ways.

This systematic review process includes searching comprehensively for all available potentially relevant evidence, and then filtering it, firstly for relevance and secondly for risk of bias. In doing so, we used structured approaches to describe and critique the available research. The review data is recorded on specialist systematic review software (EPPI-Reviewer 4) to enable transparent and accurate analysis. Details of the protocol employed in this review were published by Stewart et al. [30].

Searches

We searched for relevant literature using the search strategy presented below. Searches were run between July and September 2013 and conducted in English and Spanish. Title and abstracts of any literature identified in other languages were translated using Google Translate, and any studies that appeared to be relevant from the translation of their abstract were translated in full and considered for inclusion in the review.

Our search strategy combined the key concepts of UA and impact evaluations. Search 1, 2 and 3 were run independently, and combined as per 4 below.

• 1. ‘Urban agriculture’ ((“Urban farming”) OR (“Food supply”) OR (“Food planning”) OR (“Sustainable agriculture”) OR (“Food aid”) OR (“Urban agriculture”) OR (“Food processing”) OR (“Food distribution”) OR (“urban food production”))

• 2. ‘Urban’ AND ‘agriculture’ ((urban OR city OR peri-urban OR periurban OR metropolis OR town) AND (agriculture OR farming OR farm OR crop OR livestock OR smallholding OR small-holding OR chickens OR poultry))

• 3. ‘impact evaluation’ ((impact OR outcome OR evaluation OR effectiveness OR trial OR comparison study OR comparison study OR non-comparison study OR social performance assessment OR impact OR effects OR randomised controlled trial OR controlled clinical trial OR randomised OR placebo OR clinical trials OR randomly OR program evaluation OR controlled OR control group OR comparison group OR control groups OR comparison groups OR controls OR control OR intervention OR evaluate OR evaluations OR RCT OR experiment* OR (evaluation OR program evaluation OR economic evaluation OR (clinical trials OR trials OR randomised controlled trials) OR (experiments OR “controls (experimental)” OR trials)))

• 4. (1 OR 2) AND 3.

The above terms have been identified and tested using Science Direct. Searches were limited using specific ‘human’ filters, and exclusively included literature published since 1980 as only isolated UA initiatives existed before then. Search terms were adapted as necessary and applied to the following electronic databases and websites:

Online databases and repositories:

African Journals Online: http://ajol.info/index.php/index/browse/alpha/index.

AGRICOLA: http://agricola.nal.usda.gov/.

AGRIS: http://agris.fao.org/.

Asia Journals online: http://asiajol.info/index.php/index.

Bioline international: http://www.bioline.org.br/.

Campbell Collaboration Library: http://www.campbellcollaboration.org/library.php/.

Cochrane Collaboration Library: http://www.thecochranelibrary.com/.

Collaboration for Environmental Evidence Library: http://www.environmentalevidence.org/Library.htm.

DAC Evaluation Abstracts: http://www.oecd.org/pages/0,3417,en_35038640_35039563_1_1_1_1_1,00.html.

Database of impact evaluations (3ie): http://www.3ieimpact.org/database_of_impact_evaluations.html.

Dialnet Database (Spanish Searches): http://dialnet.unirioja.es/servlet/buscador.

Emergency Events Database: http://www.emdat.be/external-outputs.

EPPI-Centre Library: http://www.eppi.ioe.ac.uk/.

Green File: http://www.greeninfoonline.com.

IDRC (Canada): http://www.idrc.ca/EN/Resources/ResearchDBs/Pages/default.aspx.

Internet library sub-Saharan Africa: http://www.ilissafrica.de/en/.

ISI Web of Science: http://www.webofknowledge.com.

Isidore: Open Access Portal (French): http://www.rechercheisidore.fr/index.

Online Access to Research in the Environment: http://www.oaresciences.org/en/.

PubMed: http://www.ncbi.nlm.nih.gov/pubmed/.

Research4 DFID: http://www.research4development.info/.

Resource Centres on Urban Agriculture & Food Security RUAF: http://www.ruaf.org/ruaf_bieb/appflow/bieb_search.asp.

Sabinet: http://www.sabinet.co.za.

Science Direct: http://www.sciencedirect.com/.

Social Science Research Network: http://www.ssrn.com/.

UNESDOC: http://www.unesco.org/new/en/unesco/resources/online-materials/publications/unesdoc-database/.

United Nations Economic Commission for Africa (UNECA) institutional repository

http://repository.uneca.org.

Websites:

Centre for Agricultural Bioscience International: http://www.cabi.org/default.aspx?site=170%page=1016%pid=2196.

J-PAL: http://www.povertyactionlab.org/.

Search engines:

Google Books: http://www.google.co.za (screened first 100 entries on title and content page).

Citation searches for the following key publications were conducted using Google Scholar [3, 5, 9, 10, 18, 22, 23, 31–39].

In addition, reference lists of all full texts were checked for further papers of relevance. Where previous reviews were available, their reference lists were searched and relevant studies were screened as individual studies.

Inclusion and exclusion criteria/screening search results

We employed a two stage screening process, where all search hits’ abstracts and titles were screened according to the criteria listed below. Full texts of all potentially relevant studies were then obtained. Where in doubt of an abstract’s relevance, we collected full texts. In the second stage, all full texts were screened according to the same criteria as below. Two reviewers worked independently on this task and any disagreements were discussed and resolved with a third reviewer.

Population

The review focused on people in urban and peri-urban contexts in developing countries, who use forms of UA. We did not exclude any group of people on age or socio-economic group. Only research conducted in countries classified as low or middle-income countries by the world Bank qualified to be included in the review (see Additional file 1: Appendix 1). Though not applicable to the review in the end, research that focused on both developing high income countries (HICs) would have been considered as long as it was possible to isolate the impact of UA on the former.

Intervention

This review searched for UA in all its forms when used as a livelihood strategy. This included growing plants to eat or sell (for example, herbs, fruit, vegetables or flowers) and animal husbandry. Urban Agriculture when purely used as a leisure activity, such as home or roof top gardens that are not intended to contribute to either food or income in the household, did not qualify for inclusion.

Outcomes

This review focused on food security outcomes as described in Figure 1, including changes in access to, and quality of, food. Studies that did not include either one of the two were excluded. Nutrition refers to both access to and quality of food; we therefore also considered studies that assess the impact of UA on nutrition levels.

As UA can lead to a change in income levels, which in turn can have an effect on food security, we considered studies that assess impacts of UA on income when the study also related to food security. Studies that address impacts on income with no link made to food security, as studies that only focus on the environmental and social aspects of UA, were excluded.

Study design

Only impact evaluations which set out to measure the effectiveness of UA interventions on urban food security at the household, individual and/or community level, as compared to the effects of not engaging in UA, qualified for inclusion in this review. Studies that did not provide a comparison group were excluded. This was defined as a second group of participants who did not receive the UA intervention. Similarly, studies that did not measure change over time (i.e. included at least two data points at any point before, during and after intervention roll-out) were excluded.

Language

Studies were not excluded from this review on the basis of language, except that searches were conducted in English (and Spanish to a limited extent). The review team had the scope to translate studies published in English, French, Spanish, Portuguese, German, Dutch, Afrikaans, Zulu and Sotho languages. Abstracts of identified papers in other languages were first translated using Google Translate and, if deemed relevant for inclusion in the review, were assessed by a team member fluent in the language of the report. We assessed full texts in English, French and Spanish.

Describing studies

As the full-text screening yielded no studies that met the inclusion criteria, no further descriptive data were extracted from studies. Additional file 1: Appendix 2, drawing on the review’s protocol [30] outlines which data the review had set out to collect from individual studies, how the risk of bias was going to be assessed and which data synthesis strategy was going to be applied to the included studies.

Results

Results from searching and screening

Searches were conducted between July and September 2013 and resulted in over 8100 hits. By far the largest number of hits was generated through CAB Abstracts (2030), followed by those resulting from searching through reference lists and citations (1236). Specialised databases such as the International Information System for the Agricultural Science and Technology (AGRIS) Database and the Resource Centre for Urban Agriculture and Food Security’s (RUAF) online library proved very helpful with 518 and 428 hits respectively.

All abstracts were screened, leaving 256 abstracts that appeared initially to meet our inclusion criteria. After removal of 30 duplicates from these 256, full texts of the articles were sought. One hundred and ninety-eight full texts were obtained; 28 reports could not be located^b.

During the second stage screening, all 198 full texts were screened against our predetermined inclusion criteria, including geographical location, focus on urban agriculture as opposed to rural cultivation, the study’s focus on the impact of urban agriculture on food security, and the study’s methodological criteria pertaining to basic principles of impact evaluations (see above for detailed inclusion criteria). Impact evaluations are “studies which can attribute changes in selected outcomes … to a specific intervention” [40]. Hence, two key characteristics are inherent in impact evaluations, i.e. they observe changes over time, and they compare changes in selected outcomes between at least two groups. In our methodology screening we therefore looked for, firstly, the presence of intervention and comparison groups, and secondly the existence of two data points over time.

The screening of the 198 full texts yielded no studies that met the review’s inclusion criteria. These articles are listed in Additional file 1: Appendix 3. Three were not assessing food security, four were not set in developing countries, and 12 were not about urban agriculture but about rural farming. The remaining 173 studies were excluded on grounds of study design, as they did not meet the criteria of having at least one intervention and one control group and at least two data points. Therefore this review did not find evidence to support or refute the theory that UA interventions increase food security in low and middle-income countries. Figure 2 below summarises the results of our searching and screening process.

Figure 2

Searching and screening results. DC: Developing country, UA: Urban agriculture, FS: Food security, IE: Impact evaluation.

Discussion

As discussed earlier in this report, Urban Agriculture is thought to increase food security either by directly improving access to food or by increasing household income, which in return enhances access to food products. In response to the gaps in the evidence-base and the general assumption that urban agriculture initiatives contribute positively to food security, this review set out to address the question of what impact urban agriculture initiatives have, positive or negative, on food security outcomes.

The review found no available evidence, which supports or refutes the suggestion that urban agriculture positively impacts on individual or household food security in low and middle-income countries.

Nonetheless, we made two important observations during searching and screening. Our searches yielded a range of studies that study the association between UA and certain aspects of food security. For instance, Maxwell analysed the determinants of the nutritional status of children less than five years of age in Kampala, Uganda [5]. The author’s results indicate that nutritional status of children, measured by height for age, is significantly higher among farming households than in non-farming households. Gallagher and colleagues [41] found no significant difference between farming and non-farming households in their analysis of food diversity and vegetable consumption among farming and non-farming households in Nairobi’s informal settlements of Kibera. Further systematic analysis of association studies will be useful in the design of impact evaluations by providing important information on the relationship between UA and food security.

It is important to note that there is a wealth of studies discussing urban agriculture’s potential to contribute to either food security or economic standing of households and communities (see Additional file 1: Appendix 3). This is particularly true for many parts of eastern and western Africa, south-east Asia, and Latin America. The majority of these, however, consist of descriptive studies with cross-sectional designs that typically take stock of UA characteristics in a specific community at one specific point in time. Other studies are qualitative in nature and therefore provide little measurable data on the impact of urban agriculture. Further examination of this pool of literature will be valuable in developing a theory of change for how UA may impact, positively or negatively, on food security. Future trials could then measure UA initiatives’ effectiveness against that theory of change.

In some instances, systematic reviews that yield no included studies, are considered as a reflection of a review question that is either concerned with too narrow a population, addresses an overly focussed intervention, or seeks outcomes that are too specific. The strength of our review was that it set out to consider urban agriculture in all its forms, so long as it is used as a livelihood strategy as opposed to leisure activity. Food security outcomes were to be considered in terms of both quantity and quality of food. Our search criteria were sufficiently comprehensive and the search was widely spread across 28 databases, as well as additional websites and citations searches.

The question of this review remains topical and important. The fact that no studies met our inclusion criteria underlines, however, the importance of critically interrogating how the impact of urban agriculture is measured in future impact evaluations. Moreover, our Spanish language searches were limited to one database. A systematic map of Spanish and Portuguese language impact evaluations based on searches in a wide range of Spanish and Portuguese language databases will be helpful in establishing the evidence-base for studies published in those languages, which we may have missed in this current review.

Review conclusions

Implications for policy and management

This review had set out to address shortcomings in the evidence-base of urban agriculture’s contribution to urban food security, by providing an evidence-based review of all high-quality impact evaluations on the topic^c. It was expected that this would assist in making policy recommendations by informing urban decision makers on the question if and/or how agriculture interventions are best used to improve levels of food security in urban areas in developing countries.

However, our extensive searching and screening did not reveal any impact evaluations that met our inclusion criteria. Notwithstanding the fact that policy-making takes place irrespective of whether or not reliable research evidence exists, drawing inferences and recommendations regarding what works and what doesn’t from studies that met our inclusion criteria only partially, would introduce bias to the review and be inconsistent with the principles of systematic reviews. The available evidence suggests that current urban intervention programmes in low and middle-income countries have no supporting effectiveness evidence-base, i.e. the research currently available does not allow for any conclusions to be made on whether or not urban agriculture initiatives contribute to food security.

Implications for research

As all systematic reviews, this review relied on rigorous primary studies that collected relevant, high-quality data. Hence, the need for improved emphasis on rigorous impact evaluations measuring urban agriculture interventions’ impact on food security as a core outcome, stands out. Ideally, these will include control groups and measure developments over a period of time at two, it not more, data points. Funding for these should be made available. Established researchers who have published broadly on the topic, some in a range of cities around the world, are encouraged to collaborate with impact evaluation experts in order to set up rigorous impact evaluations that answer key questions around urban agriculture and its food security impact on households and communities. In addition, existing and new qualitative data should be drawn upon in developing theories of change, which in return will aid the conceptualisation and analysis of impact evaluations.

In the systematic review community worldwide, systematic maps have become increasingly common in recent years [42–44]. This review alludes to the value of conducting a systematic map prior to engaging in a full systematic review. A map lays out the available evidence and provides an overview of the characteristics of the evidence-base. It can provide a summary of methodologies typically employed by researchers in the topic area and guide the exact focus of any subsequent full systematic review. This is particularly valuable in a topic area that draws on research from various disciplines, or interventions and outcomes that have seen little consideration by systematic reviewers. Members of the systematic review community, particularly those who conduct reviews in areas not historically covered by systematic reviews, are encouraged to consider the value of compiling a systematic map for the review question at hand prior to developing their full review protocol. Producing a systematic map for this project prior to engaging in a full systematic review would have outlined the characteristics of the existing evidence-base. In return, it would have presented the review team with suggestions for changes to the inclusion and exclusion criteria of the full review.

Lastly, we suggest that this review be updated in due course so as to provide answers to the important question of what the impacts of urban agriculture on food security are and to provide insights for practitioners and policy makers. Meanwhile, compiling a systematic map that details the currently available research on the association between agriculture and food security and which outlines the characteristics of urban agriculture and food security research, will provide a valuable tool for both practitioners and researchers.

Endnotes

^aLivelihood is commonly defined as comprising “the capabilities, assets … and activities required for a means of living” [45].

^bOur search for these full texts included inter-library requests, searches on a range of international academic databases, and searches through google and google scholar.

^cThis refers to all high-quality impact evaluations that would be identified through the review’s search strategy as outlined above.