FINISHED COPY
NINTH ANNUAL MEETING OF THE
INTERNET GOVERNANCE FORUM 2014
ISTANBUL, TURKEY
"CONNECTING CONTINENTS FOR ENHANCED
MULTI‑STAKEHOLDER INTERNET GOVERNANCE"
05 SEPTEMBER 2014
11:00
WORLD WIDE WEB FOUNDATION OPEN FORUM
MEASURING WHAT AND HOW
CAPTURING THE EFFECTS OF THE INTERNET WE WANT
***
This is the output of the real‑time captioning taken during the IGF 2014 Istanbul, Turkey, meetings. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the session, but should not be treated as an authoritative record.
***
>> MODERATOR: Let's start. Welcome to the open forum organized with the Web Foundation.
I'm Khaled Fourati, project manager, managing the Web Index Project and the Web Foundation.
I have asked a few colleagues and friends to join me in this panel with the idea we discuss challenging and opportunities around the measurement of the open Internet environment.
On an annual basis there is a web index that tries to capture how people use the Internet, utility of the Internet, the effect of the Internet and it is a multi‑composite indicator and there are a few methodological challenges and ways to improve it over time so that you better communicate it so that actors, policymakers can use it, so forth. The speakers that join me here today, they have a wealth of experience and different measures around the Internet.
I will start with Stefano De Sabbata from the Oxford Internet Institute. I specifically asked him to join the panel. I found that the Expert Internet Institute does a great job around visualization and they do a lot of interesting work on capturing data and putting it out there so that people can understand and sort of snapshot what's really happening. I wanted him to share that with us, how they do that, the type of work that they do.
The professor here that represents the technology sector here, a former Minister of Science and still policy making in terms that he put together through Mozambique, he's among the pioneer in the Internet in Mozambique.
And we have Robert Guerra. I said earlier that you have many hats, his is the University of Toronto. He's also I would say helping Civil Society organisations around the world to better understand how they can manage their information online, circumvent any censorship and surveillance, so forth. I wanted him to speak about that, that elements and topics.
My friend and colleague, Yacine Khelladi, who is the Alliance for Affordable Internet. And the Alliance for Affordable Internet, this is the second edition of the affordability index, I thought he would give perspective on that.
And Alison Gillwald, Dr. Alison Gillwald, who has been involved for some time now through her research, ICT Africa network. It is the multi‑country network in Africa, now around 12 countries, but started more than 20 countries and they have been doing a lot of work on primary data collections through households and survey. So analysis and supply side as well as analysis and bringing that together to basically inform policy‑making processes.
So as you ‑‑ as you see, we have a wealth of experience and knowledge, and I wanted to basically structure this session as a forum for discussion.
It actually is interesting, the different people from different perspectives that work on different dynamics of measurement of the Internet or the openness of the Internet. It is unique actually because usually you don't find people who are working in different angles coming together and discussing a holistic approach. Usually people are talking about access, content or talking about impact, but they don't come together in a panel to start discussing this.
The idea when we started, when the Foundation started, the web index ‑‑ sorry, I joined the foundation a few months ago ‑‑ it is that it is not only about measuring access, universal access. We have to understand rights, rights on the Internet and how the experience of the uses in the Internet. We also need to understand local content and to understand how it is effecting us all. That's the idea of developing that multi‑composite indicator.
Let me open it to the panel for discussion. The first question I'm going to pose to you is telling me a bit more about how you go about measuring the openness of the Internet, the type of work that you do, how this question basically fits from your perspective.
>> STEFANO De SABBATA: As you mentioned, in the last year, along with a colleague of mine, at the Oxford Internet Institute we have tried to measure various ‑‑ various attributes of the Internet in particular concerning the geographies. We tried to understand and analyze Internet data from different perspective, and the first one is the accessibility of data. So where people have access to the Internet, how, why do they have access to the Internet? We tried to move beyond that, trying to understand how people participate, how much they participate in the Internet, and as well as how well different countries are represented on the Internet, what the Internet talked about in terms of geographies.
And in general what we have seen is that there are various large differences. So in our understanding the information divide on the Internet is even larger than digital divide itself. The participation, the representation are a reinforcement on the digital divide. In general, that's what we have found out until now.
I pass over the microphone.
>> MODERATOR: You want to speak to this question or ‑‑
>> VENANCIO MASSINGUE: The same question?
>> MODERATOR: The same question, please.
If you want to speak about, for instance, the involvement, the involvement of your center in data collection and the development with the statistical offices?
>> VENANCIO MASSINGUE: Thank you very much. Thank you for inviting me to be a part of this panel.
I tell you, before I went into ministry, before I was a vice director, I was more involved with the actual doing and then I became a sort of policymaker. Now I decide I better come back and do the job myself.
(Laughter)
>> VENANCIO MASSINGUE: One is to set policy, the other is to get results. What I'm going to tell you is what I have learned from the policy side.
In general in Africa we don't have enough statistical information in the area of ICT, in particular in the Internet or on the science technology innovation, on ‑‑ we ‑‑ on innovations or so. We don't have enough information. When I say "we" I mean now me and now before, when I was a minister, meaning that some decisions we have to make will require more accurate data. For instance, if you move around, you ask, oh, how many Internet users do you have? The first thing we do, we learned to look at ITU statistics. Yeah, this is a statistic.
So in one meeting ‑‑ it was very interesting ‑‑ in one meeting three months ago, one person was saying you know, the coverage of Internet in Mozambique, it is 4.5% of the population. Someone else said no, no, 6%. Then you ask what is the source of information ‑‑ no, I saw an ITU. I saw that. I saw different supplies of information.
Of course, the data, it is difficult for you to make the appropriate decision about some of the things you want to do.
My ‑‑ the reason and interests, again to come back to this issue is because what we would very much like to see is how can we establish an entity that work with the national statistics to obtain a more precise data gathering and processing and keeping updated information about what is going on. The elements here of capacity building and technology usage for collecting information. Storing, for making it available. We're also looking for partnerships
The second point, I'm very much convinced that all of the current information that is made available to decision makers will be used, that it can be directly through the national statistics or through those entities that could participate on the production of the real data. So my recommendation also will be that we bring more partners to help establish the technology to be able to generate the necessary data.
Finally, this is my official talk, a gap I have noticed very strongly, I visited all these institutions trying to see how you make a link between technology developments and the GDP. It has been very hard to find this link. It is very complex compared to what I think maybe would be a logical link.
How when you establish, for instance, mobile software for people to be able to sell more products? How do you use this as in a contribution to the GDP? I don't know. This is also to see if someone is doing something on this in terms of statistics and the data that they incorporate on the forum for the calculation of the GDP of a country. That's what we find.
Thank you
>> MODERATOR: Thank you. You raised a few issues there that I think we'll have to come back to them.
At least one that I noted, the proliferation of data, the quality of data. Then what perhaps the forum makers are about to make his or her point, is around the impacts. So we'll come back to those.
Robert, your turn
>> ROBERT GUERRA: Thank you very much for inviting me on this panel.
What I'll do very quickly, I'll maybe say a little bit about myself in the citizen lab and try to get to your question.
You introduced me as being with a Citizen Lab, for those of you that are not familiar with the Citizen Lab, the Citizen Lab is a unique multidisciplinary research laboratory at the School of Global affairs, University of Toronto. We do something unique in this space in that we bring together experts that are familiar with the issues of ICTs, international security and Human Rights. There is a variety of different activities that we do there. I'll touch on them in a moment.
I'm also a member of ICANN and Stability and Advisory Committee and been involved in some of the studies that they have done, some of the reports that they have done on the consequences of Internet blocking, and have been a long time participant the IGT and predecessor, the World Summit on the Information Society.
The Citizen Lab is a pioneer organisation that's been measuring the Internet openness and by Internet openness. It is how we see that term as evolving. One measurement is, you know, net openness, accessibility, number of users. What the Citizen Lab has been doing, it is evolving over time, trying to measure for us Internet openness. When we started in around 2002 we were one of pioneering groups together with the Brickman Center and others that we started using research‑based methods and analytical tools to basically determine the extent of censorship and blocking that was taking place around the world.
There are a lot of reports at the time from users in China, elsewhere, of what was happening. What we did, we built technical tools and collaborated with the users in the country, one to determine the extent of the problem, and then due to a technical interrogation to find out the nuance, the type, and we have seen an evolution of blocking. That it was originally a few set of countries, and it evolved now to almost all countries in the world blocked, not just Turkey, Canada, U.S., many others.
What we have seen with evolution ‑‑ I won't get into too much of the details ‑‑ it has shifted in regard to far more extensive and advanced tools are being used by ISPs and governments around the world. Not so much censorship and blocking, but more so now targeting surveillance, digital attacks against governments, businesses, against actors such as Civil Society.
A lot of the work of the Citizen Lab over the last two years, we studied extensively working with partners in countries using what's best described as very advanced digital forensic methods to actually analyze, one, whether the hypothesis, attacks were taking place; two, attributing the commercial products made in Europe, Canada, elsewhere that are being used. And what we have seen is unfortunately an advancement of the sophistication of the measures, and so we measure it through advanced technical measures, stuff like that, and what we have also seen is particularly important now and I think the session previous to this one touched on it, to measure the Internet openness is also having a dialogue with providers to see how transparent they are in terms of throttling and in other types of issues that they do.
So I would say that to get back to your question, in terms of measuring Internet openness, the work we have done, what we publish, it is through research‑based methods to get to one of your points, which is to use research‑based data to inform policy and inform actor as far as the threats, blockages, the best strategies to move forward.
I'll get into more details, but we'll let others answer.
>> MODERATOR: Thank you.
Yacine Khelladi
>> YACINE KHELLADI: Yes. I'm pretty much new in this Alliance for Affordable Internet and Web Foundation Team. I'm just starting to participate in the basic data information ‑‑ more than the data, but the collection on the policy regulation and the environment in which each country ‑‑ I mean the environment in each country, which favors or not policy reform that would lower the cost of access to Internet and Broadband and better penetration.
This is the affordability report of 2013 has been released I think ‑‑ Emily, when was that? May, March? In December. Okay. Before the end of the year, you will have the 2014 Agenda. The scope has been expanded. The number of countries that are in the affordability report, it has been expanded.
The point though, it is a tool both for informing but for making decisions, informing decision makers, it also has to be ‑‑ I have been involved in the process of three countries, you know, getting ‑‑ interviewing people in the countries, really to understand the situation. It really helps a lot. It helps me a lot. Helps identify the key challenges, key questions that change a lot from country to country. It is very difficult to benchmark ‑‑ you can benchmark statistical data, but it is very difficult to benchmark how the law was designed 20 years ago with one that was designed 5 years ago, how the local, legal system, how in countries that have 7 regulators for ICT and others you have one, in some countries you have to go to the judge to enforce regulation. It is quite difficult to ‑‑ it is a tremendous effort to benchmark all of this together and have an index. This is a tremendous work.
I have been working in ICT for 20 years, trying to measure things for ICT for 20 years. The question is how beyond a simple number of access, numbers of statistics like the ITU statistics, the number of computers, all of that, how do you measure the impact of that? How do you have a sense of this is really making a social change empowering people? It is so ‑‑ it can be used for so many things and different ways and empower different people. It is difficult to measure. That's when you see that the traditional evaluation ‑‑ let's say framework, the methodologies fails. You're not just adding up the number, but looking at the process, the empowerment.
That's why for example I was telling you this morning in our breakfast I learned with IDC the outcome methodology, I liked it a lot. I was ‑‑ I'm not looking at impacts, but just looking at how much people change their behavior in their day‑to‑day things when you introduce ‑‑ change one of the factors and this is introducing ICT, for example, that's very important.
Another question is for who we're doing this. Are we doing really this for ‑‑ who needs to be informed? Who makes the decision? Not only the decision, political ‑‑ policymakers, decision makers, the people themselves, the organisations, the NGO, the communities. We have to think about for who we're doing this evaluation. Who is the actor in the evaluation? I know that the evaluation, the monitoring, the data, it takes a lot of energy and time, and we don't have time to do the things. We're in the morning looking at the projects. You ask me to evaluate, and then to attend more workshops to evaluate and understand what I'm doing at the same time. It is a problem of understanding how important it is to measure, evaluate and to be able to look at these things.
Measuring processes, measuring empowerment, measuring environment, it is challenging. We're trying to do our best to produce. I invite everybody to stay alert when the new reports come out which will highlight how much policy and regulatory reform can impact access and affordability of access.
>> MODERATOR: Thanks.
Alison, your turn. I wanted you to perhaps speak about both the demand and the supply. I think Korea has a breadth of knowledge and expertise in that. Let's start with those, how you combine the two and take it from there.
>> ALISON GILLWALD: I have a few slides here, just because I think it can ‑‑
>> MODERATOR: Is it ‑‑ do we have her slides?
>> ALISON GILLWALD: I think it captures it much more if you just give me a ‑‑
>> MODERATOR: No. Not this one.
>> ALISON GILLWALD: Perhaps as they get it up ‑‑
>> MODERATOR: No. It is not that one. It was mine. I'm not using it.
>> ALISON GILLWALD: I'll speak a bit about research ‑‑
>> MODERATOR: Is it measuring worldwide ‑‑
>> ALISON GILLWALD: Yes.
Research ICT Africa grew out of a need for evidence on which to build evidence‑based policy in Africa. I started research in ICT Africa after coming out of the regulator, the first regulator in South Africa in 1997. Having just experienced the fact that we had no Indigenous data, no Indigenous knowledge really about regulation, we were working with very sophisticated, imported frameworks from the European Union, somewhere else that really didn't bear any relation to the maturity of the markets, competitiveness of the markets or capabilities of our institutions and regulators.
Initially we started off trying to gather together supply‑side data that was more up to date than the ITU data, and obviously that was all we had in this mobile environment and where mobile up‑tick was happening and we were dealing with data from the ITU that was two, three years old. What happened, we were working with listed companies, reports, getting the data up quickly, but we soon realized that it was not the regulatory oversight of the operator supply‑side data. It was ‑‑ it was very misinformed and misinforming. One of the earliest issues we worked on that's now sort of common knowledge, but we worked with closely with Learn Asia in the ITU on ‑‑ it was trying to understand subscriber numbers from our mobile phones. Operators were giving at that time even magnitudes of scale larger than it appeared people were using them on the ground, not these 125% when we all know people that don't have phones.
The issue of number of SIMS arose. The question was how many are there? It raised the issues for the need for demand‑side research. We had to understand the market. It was to get on the ground, go in the field, actually speak to people about what they were using, what they were not using. What's significant about the reports compared to a lot of the reports of this nature that are done in the developing world and very much done with supervision from the north and people operating on the ground in the south, a disconnect between the two. It is that this research really looks not only at those connected, the bottom of the pyramid, not looking at the use of women in, you know, Uganda women, a center especially designed for women. It does nationally representative surveys, nationally representative surveys which then can be sliced and diced as you need the information allowing for rural, urban disagregations which cannot be blocked for mobile in any other way than on the research.
You know, people have argued now with the registration of SIM cards. We don't really ‑‑ that's too big of a problem anymore. You know, we all know that the disaster of SIM card registration on the continent has taken millions offline, and it really limited access for those that can't ‑‑ who don't have identities, that don't have postal addresses, that kind of thing. I have at least 6, 7 people illegally registered on my name. I long to be taken to court on an intelligence issue on this so that I can expose how accessing these cards and metrics, things like that ‑‑
>> MODERATOR: Nobody would dare
>> ALISON GILLWALD: This is not increasing it. This is the research we're doing.
Is that presentation up? Can I just ‑‑ can I show you some of the results of that?
So I think the important thing, I think some of you have seen that John has produced that lovely map of the world which shows that he's pinned all of the points on the web shown. You know, Africa is a big black hole and, you know, spots around southern Africa, North Africa, and then, of course, it is lit up in the north and in parts of Asia. Our research is trying to understand what's happening in that big dark patch. Understanding the linkages between the big dark patch in Africa and what's happening in the other parts of the world.
Understanding the reasons for people's marginalization is important in policy intervention if you're trying to increase access. I think an important thing about this research is that it is not only ‑‑ it not only identifies lack of access, what we found, having identified issues around the lack of access is that the issues were far more complex than that. Once we move beyond the more descriptive kind of data which showed, for example, in relation to gender, to women, that, you know, women may or may not in certain circumstances have more access to certain things than men and in many countries women have more access to mobile than men, if you begin to look at the usage patterns, what they are expending on it, it is far more uneven. The issues of inequality, it is raised beyond access. We see this particularly with the Internet when we begin to look at data usage, that kind of thing. The correlation between accessing usage is really between income and education and gender is ‑‑ it is only a gender issue because women are concentrated in lower income and lower educational groups.
I was just talking because I can't actually get this to move on.
Okay. Quickly I'll run through some things very, very quickly here.
Just to say that the importance of a household survey ‑‑ in order to understand the relation of communication spends but the importance of communication and the willingness of poor people to spend on communication, it is really some indication of how important it is to the social and economic well‑being. You see power remains a major issue.
The other point I wanted to make quickly, if we are talking access to information, radio still remains the most important form of information in the continent. This is changing. It will change as we increase in Broadband.
We're giving away spectrum to people that actually are not connected to Broadband, they'll have nothing. Just to say, the fixed lines, we're talking about mobile penetration. If we talk about the network access, the access network, we're talking about mobile. There is big issues around fiber, black hole, international prices coming down, that's very important. If we talk about connecting the last billion now ‑‑ we're talking about mobile ‑‑ we can see the limited number of people with working computers, this obviously changed with the connectivity provided by mobile phones
And then I'm taking you very quickly because you can look at these, this is all online in the public domain. This research is done, being conducted, the research, every three years over the last decade due to the extraordinary support we have had from IDRC and of course the last few ‑‑ sadly this research, in the next year will only be conducted in three countries. We have gone from 17 effectively to 120 and then only to 3 next year.
What you see from here, for example, it is that the real take‑up with mobile is extraordinary, and that's what we're ‑‑ we'll see a similar pattern as what we saw with voice. I want to say, you know, basically doing this national representative research allows you to slice and dice it in a number of ways.
Social networks is a driving take‑up. If we look at ‑‑ if you look at the gender data ‑‑ which is important about moving beyond the descriptive statistics is the kind of modeling allowing you to really see what the correlations are, and what the linkages are and trying to identify points of intervention ‑‑ often it is depressing. Putting up the women telecenter will not solve the problem in terms of ‑‑ this is intergenerational, long‑term human development issues that need to be addressed.
I want to quickly ‑‑ I know that Khaled Fourati is wanting to move on ‑‑
>> KHALED FOURATI: I was going to say with you, to follow‑up with other questions, could you speak to the supply‑side data?
>> ALISON GILLWALD: Yes.
Having worked on the ‑‑ simply the access supply‑side data, what's apparent as I said very quickly, it is that in terms of stimulating demand, pricing was a critical issue. That the pricing, particularly mobile pricing, was opaque, and in Africa we started off doing prepaid mobile voice indexes. The one point I want to quickly make about indexes, seeing it is the web index panel, it is that I think one needs to think about the value and the purpose of indexes.
For example, with the work we have done a telecommunication regulatory environment survey, a perception survey of operators and it uses the main five WTO categories, so including universal service, tariffs, universal service, spectrum resources, et cetera. Civil Society, operators, everybody but the regulator and government because it is about the regulatory environment, they're able to assist across the categories. Now, when we ‑‑ when we originally started to do it, the purpose was to ‑‑ it was a thing if we can show that countries were being poorly perceived in terms of regulatory effectiveness, then we could challenge them. In fact, because these things are so nuanced at the national level, we found that the studies actually were far more interesting for a country at the national level to see the progress it is making in universal, in spectrum, now on interconnections and competitive issue, et cetera, rather than compare them together in South Africa we have a critical and vibrant and challenging environment, everybody criticizes everything. In South Africa, it is ran poorly, you cannot compare it with another environment where there is less criticism. This needs to be understood, used carefully.
The supply side ‑‑ now we have moved to Broadband ‑‑ this is a challenge because of the complete bundling, integration of voice and data which makes it very difficult to assess how many data users there are and to deal with the issues of prices. For example, in Egypt right throughout the political crisis they had continued to have extraordinarily dynamic pricing product, et cetera, you can just buy air time and you make up voice and data, whatever you want with that. It is not even that operators in advance know what they're selling and what they're getting back.
Just to qualify this, the pricing issue, we have ‑‑ you know, we had put up several indexes and we found that a country like Cameroon, for example, was the cheapest country at that point ‑‑ I think it was 15, 17 we looked at. We have been looking at that, saying what is Cameroon doing right that it is so cheap? What we found, it was these poor operators that have been blocked in the evolution on 2.5 standard quality, they were offering dial‑up quality because the 3g license had been granted as a monopoly to Viatel and essentially not becoming operational. The Cameroon, had just fallen back after being one of the leading Internet countries in terms of aggregated access, it fell back entirely.
So the qualification of these pricing and things, it really makes a difference. Otherwise, the evidence is not accurate. The main qualification, the last point I'll make on this, it is that we have now looked and taken this pricing data and looked at it against the quality of service data. At the moment outside of South Africa this is just ‑‑ in South Africa, it is testing. Some of the price, they're higher in certain countries, they're for much greater quality. For better speeds, everything, we have latency problems,
We have all gone through London, whatnot, the actual price on a pure price basis index it is really mis‑informative, non‑informative. I wanted to say this is the research that's being done, and it is all in a public domain and the databases for researchers, also available, but they require a little more coordination and we're lean and mean on the ground.
It does require to give us time to require to us work with you on that data
>> MODERATOR: It is a nice segue of the next question, to the next question.
You started to talk about the challenges. The technological challenges and the types of indicators that you could combine, not combine for what purpose, so forth. That's specifically what I wanted to ask ‑‑ perhaps Robert, if you could take it off from here, in terms of the challenges that you're facing at the ‑‑ to collect this data, this information, both methodology and also in terms of supporting the capacity development on the ground. You don't necessarily do all of the collection yourself, you're basically collaborating with partners. I would love to hear more from that and if anyone wants to jump in on this, then I'll give them the floor.
If you have a follow‑up I'll give you the floor after this.
>> ROBERT GUERRA: Some of the work I forgot to mention earlier, some of this was published under a study, a collaboration we do called the Open Net Initiative. If you Google that online there are three publications put out over I think six different years on a variety of different countries. So that's online as well.
For challenges, it is a hard question. I'll try to tackle it in different points.
First of all, for some research that we do, we're lucky we have partnerships in particular countries, it is a trusted relationship required, particularly in some of the ‑‑ I'll get into that a bit more,
The ethics, the protocols that we use, we have developed them, but it is different if you're measuring kind of Broadband measurements, network measurements. If you're trying to measure Internet blocking from a particular country, those types of issues as a researcher you get access to a lot of user data. One of the things we have done a bit differently than many others, we actually have a formal ‑‑ we go through approval at the University and then in turn we have a formal type of relationship in terms of what is collected, who on the team has different access to different information, and going through that very vigorous approach that takes a lot of time but is particularly important and we have shared that as well.
In terms of challenges, where we get the information, I think we have had a lot of ‑‑ I guess we have just been lucky that we have had relationships. For example, one of our earlier publications that got a lot of media attention, which was related to a malware effecting Chinese groups around the world. It was really thanks to a very special relationship that we had with the Office of the Dalai Lama. If you want to have that trusted relationship with the stewards of the data, the researchers, then if you want to go to a more global pattern, a regional pattern, then you develop those trusted relationships and building up that in other parts of the world. It takes a lot of time. Where we are, kind of now, it is that we have a lot of expertise and research we have done in Asia and in the Middle East.
As I commented before we started, it was more of a challenge with the technical researchers we can work with, groups on the ground, Africa, particularly Latin America. When we talk about the pattern of increased digital attacks against organisations around the world, it is hard ‑‑ we don't want to generalize. All we want to say is that, and then we have questions from different parts. I think it is one ‑‑ it is also the retention of staff that has special, unique skills that can get orders of magnitude, more funding if they worked in the commercial, the private sector, finding, recruiting that; two, maintaining a real diverse intersection of both the technical, non‑technical on the team.
Something that we do that ‑‑ I don't get into the funding issue, but more of how we share our data with the community, we put out the data for the initiative. It sits there, but not a lot of groups are using it. We try to encourage the data that's been shared and the frameworks for others to replicate the work as well too. It is very sophisticated and things like that.
Just to finish up in terms of the challenges, I think it is trying to develop for the IGF. I think it is some of the work we do, it effects the whole system, in trying to find other actors to be equally interested and that's our big challenge in Canada. We were a leader on this issue to try to get companies interested and to get the Canadian government interested is a big challenge. So we think it is an issue of concern in getting the attention of those that make policies, it is crucial.
>> MODERATOR: Thank you.
>> AUDIENCE: Thank you for your interesting presentations.
I have a couple of questions: My first question is what are the primary recipients of your data collection and evaluation efforts? Policymakers, regulators, researchers? Have you seen any cases where the regulators used your data for their market analysis work for example?
Finally, my last question to Alison, your cross‑country comparisons in terms of price, quality of service benchmarks, have you seen the cases where consumer organisations use this for consumer right issues, to lobby for?
That's an issue, for example, in my country where consumers are continuously complaining about the quality of Broadband service they get from the operators, and this might be a very useful data resource for them to have in front of the operators and the regulator itself to improve the quality of service.
Thank you.
>> MODERATOR: Alison Gillwald, Robert Guerra, whoever wants to jump in.
>> AUDIENCE: I come from Armenia. I head a Civil Society organisation and simultaneously I'm an ICT consultant
>> MODERATOR: Perhaps if I can pick up on his questions on what you said, Robert, on how to communicate to the Canadian government and to the policymakers. If you speak about the strategies you use to get the message out there, to simplify the data, the indexes, so forth, to have them more in a user‑friendly way, or other type of strategy that you have looked at.
>> ROBERT GUERRA: There are multiple recipients of the reports that we do. First of all we ‑‑ we publish in a traditional academic sense, in peer review journals, inform, actually organize seminars and events quite often.
What's different in a lot of work we do too, we actually ‑‑ most of our reports actually over the last two years, they have been of considerable interest to the mainstream media. We have ‑‑ I can't count right now the number of above‑the‑fold articles we have published, reports we have published that have been published in the New York Times, the Washington Post, press around the world. So it is working, I guess, with having media contacts,
Getting to your question as well, you can take what's a technical report and not necessarily have the staff, but the researchers, they're on the ground to speak to the media to basically have the impact of the changing landscape of Internet controls that are taking place and what we believe is the effects of our reports, what they have also ‑‑ not just the recipients, but what they have been able to do, some governments may or may not be listening, there is a ‑‑ definitely communities trying to support, certain communities, Human Rights defenders, others, they have the groups safer online. What is important there? There are assumptions of what the risks and threats are and what we think is that our data, the research we do, actually are evidence‑based data on the actual threats and issues that are taking place, and then that's helping the donors in the space tailor the program, their supporting of the programs and what we have seen, the sophistication of the issues, that are being faced, it is not just about having fast Broadband in a particular country, but if users then want to partake in particular type of activities, that may have sensitive, other types of information, we don't necessarily do advocacy ourselves but what other groups have done, they have taken our data and use it had to help educate users in terms of the options, risks that they do. Not so much in Turkey but Mexico, but some of our reports triggered legal proceedings against companies or governments.
We ‑‑ you, the recipients are the public, but we have done it in a vigorous way to be used by a variety of those which wish to advocate, but using the technical data that's there.
>> MODERATOR: Do you want to respond?
>> ALISON GILLWALD: Thank you. Perhaps I am in the fortunate position of having just undergone ‑‑ or unfortunate, of having undergone a vigorous independent review of 10 years of funding of IDOC. Which was actually interesting for those of you interested in evaluation. It was ‑‑ it was a major research project to try to establish ‑‑ to try to establish the theory of change under apparent success of dissemination and policy influence. The whole evaluation is done as a research to policy influence analysis.
You know, when we started this project in terms of trying to get funding from a number of different aid groups, you know, people really expected demonstrations of a silver bullet. You gave us this money, we went, we did this, boom. There you are. I think the most significant thing about the report which was done on behalf of IDRC was the demonstration that was the consolidation of the ‑‑ of work from outputs, of credible outputs over a decade that's allowed us to be effective, to withstand criticism of operators or governments, wherever the people that don't like that research.
I can say this: What distinguishes our work compared to the global work that's done, it no ‑‑ it is not my doing. It is done locally, on the ground, by people from University, research institutions, et cetera on the ground that have a good understanding of what's going on. In that degree, it is reasonably uneven as well. Those different individuals are more embedded in the politics, influenced in the different countries and so have different successes.
I think the hallmark to it all is that it is independent, vigorous defensible research. In this way it has informed ‑‑ it has been publicly cited and used as evidence for a number of processes, external standard settings ‑‑ to highlight one, it is close to us, the silver bullet for us, the work on mobile termination rate reductions across the main countries that have done it in Africa, and the effect on retail prices and that dramatic cut in retail prices and this really turning the conventional wisdom of mobile termination regulation from Europe on its head. We were able to demonstrate without any exception that a reduction in mobile termination rates did not produce the effect that operators claimed that retail prices didn't go up in any jurisdiction ever, and that you got an expansion of subscribers and sometimes the incumbents, revenues actually went up from termination because of the traffic going through the networks. Times they were down, you know ‑‑ sometimes they were down, a real win/win for everybody. That was possible because of our mobile termination rate data collection on the ground because often it is not ‑‑ people are not aware of the processes, et cetera. It is local people on the ground giving us constantly the developments in terms of mobile termination rates, and then linking that to this extraordinary resource intensive and dreary data collection which we have collected all of the product of all countries quarterly from all of the operators.
That takes me to ‑‑ I want to say that you can use that pricing data. It allowed us to act as a friend of parliament for parliamentary hearings on the cost of communication because the parliamentary committees don't have the expertise, whatever, to develop questions or to provide some counterpoints to the stakeholders that have been coming before them. To quote the South African parliamentary board committee head, lying to us for a decade on what's happening with the pricing and cost of communications.
Just on the consumer points: We have an enormous challenge in Africa with consumer awareness and even our regulators, a lot of our regulators, they'll say my job is to ensure that there is competition and investment in the country. People will have a ‑‑ they never will say my job is actually consumer welfare. To do this, I'm showing this is competition, whatever. They often are not mentioned at all. There is no recourse to them. We tried to sell an Excel‑based ‑‑ people don't want to know where they are with the country, but what package they should buy for their needs. We try to put up a basic thing for consumers to use, to give it to the South African regulators, to give it to them so they can put it on their website and to possibly do some analysis on that assisting the consumers. Honestly, you know, you need a Ph.D. to work this stuff out. Basically, they are not really interested and concerned about the liability issue, the accuracy of the data and if people are in contracts, whatever. It is a challenge for us and that's just not been pushed hard enough on
>> MODERATOR: I will go to ‑‑ anyone want to address this question? You want to respond? I will finish the respond and then go back to you.
Do you want to use the slides now?
Can you quickly put the slides for Stefano De Sabbata up, please? I know you have the visualizations.
>> STEFANO De SABBATA: My expertise, it is in the ‑‑ can I ‑‑ thank you.
I have been also trying to communicate this kind of information and outcomes of our research to the broad public. Even with my background, I really think that using this organisation, it is an effective way to communicate especially to the public if you want to communicate with experts then it is slightly different. For the public, it is more effective as usually people don't like to look at tables and numbers. However, especially the west, we are usually ‑‑ visually, we're overstimulated. A graph, a simple graph or bar chart is usually not attractive. What we're trying to do is to develop ‑‑ let's see if I can manage ‑‑ to try to develop more customized organisations.
For instance, this is showing Internet population as detailed by the World Bank in 2011. Each country is precise in the Internet user and the color code is the person data. You can clearly see how Africa almost disappears and how China is basically a third or 20% of the overall population here, and how distribution of the color is centralizing Europe and North America.
We try to use these visuals to better communicate information. I would like to also jump back to one of the issues that's been rose, trying to go beyond with our institution with China and the national statistics, trying to collect other data.
For instance, this is just a mix of two institutions: China, here we're mixing the Broadband cost from the ITU and the gross national income per capita from the World Bank trying to see how affordable is the Internet cost for the population. In our research, we're really trying to go on the ground in another sense, trying to go on the Internet, trying to collect data there rather than institutional channels or on the ground as many other projects are doing. We're trying to actually go on both grounds, on the Internet, on the ground, trying to combine these two visuals, these two understandings.
For instance, we collected data ‑‑ for instance, this is the leading website for collaborative software, so tying to share your software. Even accounting for the Internet population of a country, not the general population, we see as I said before how the informational divide and the production divide, it is far larger than the digital divide itself.
We can see that most content is produced in the U.S., in Europe, even considering the large population and produced in Africa, for instance. Unfortunately, coming back to this, looking carefully, you can see the two globes, they're surrounded by a pie chart which represents the number of users and number of uploads per country. Here we're trying to have more information standards so that you can have one overview, the first look, then going more to ‑‑ can I just show this one slide? For instance, this is the coverage of Wikipedia, the central hub for knowledge discovery for most people on the Internet. As you see, the informational divide here, the representation divide, it is enormous. Most of the content is about the ‑‑ is about Europe, it is mostly centered in Europe, over 60%. I think this visualization can strike really the general public. This also connects to a principle that's been suggested by my colleagues in a recent paper, that's the principle of growing information poverty, where the content in the Internet leads to the creation of further content. If we have content of Europe, this leads to more creation of content of Europe because of the mixing of data, translations. You can see France there, there is a lot of content about France, because the content about France has been translated sometimes automatically in many other languages. We have a lot of content about France for that reason. I think this is a good way to communicate at least to the broad public.
>> MODERATOR: Thank you.
Yacine Khelladi, you wanted to respond as well?
>> YACINE KHELLADI: For the use of the data, it is interesting. The research, it is interesting. The question also, the question was asked how do I link ‑‑ how do I link this very basic Telecom access indicators with GDP, with wellness, with how ‑‑ how the social or economic situation of the individuals, it is quite a complex question. And I have been reading in the plane coming here, information published, a book published, the information life of the poor, I think there are several ‑‑ I think you collaborated with that. I was surprised to see several studies of several outcomes of this research linking that. So this is for policymakers and for people that do advocacy.
If you want to go to the broader public, you would look, for example, there is another funded project in Uruguay a few years ago about how to use the social network to get this research, pure research data, outputs, and to use it for mass communication and influencing the public opinion like it was done in Mexico when the regulator was about to change some regulation that would give more market to the incumbent ‑‑ in fact, and thanks to the use of social media, and research, it was clearly demonstrating facts, we have had that, and they were able to block ‑‑ to influence efficiently the public, the Twitter, Facebook, all of those things, the government. The parliament stepped back on that measure, something like that. So there are many ways to use the research and being able to do reality change, to change reality directly to public opinion or to inform policymakers with evidence and facts that they can convince them to change. It is very important.
Often researchers tend ‑‑ we ‑‑ the researchers, we have to rely with the communicators and people that are able to translate that. And it is very important. Very effective, I think.
The visualization, the graphics, that you can easily put in the newspapers, on Facebooks, whatever. It is quite important if you know how well to produce them.
One small thing I want to throw, the data from the operators, for me, it is a critical thing. I don't see how ‑‑ the operators, they must know what the people use, how they use it. How much they use ‑‑ how many time they spend, how many hours they spend on this, that. They cross that with the profiles. They have the profiles, they know the gender, age, they probably know the kind of ‑‑ if it is a private, a business, something. There is a ‑‑ I think the universal data here that we're not having access to, right? From the operators themselves, that's a wonderful thing to have.
>> I lead research. I'm from Yemen but work in Sweden. Ironically enough, the Sweden tops the list, and Yemen, at the very bottom. I have a view on what lies ‑‑ what the challenges that lie ahead.
Let me say, it is rather important that we understand that there is a huge gap between countries. I see this firsthand by living there and also living there.
There should be more initiatives and I take this opportunity on the opportunity, appreciate the cyber steward program, the program bringing activists and researchers, people involved in the Internet from different parts of the world to work together to introduce research, ideas to bridge the gap. However, I think that we still need more of this. While there is research such as the open net initiative that was pioneering, a solid part I referred to in my Ph.D., we need researchers on the ground in developing countries. One ambitious idea is to perhaps build bridges of understanding, collaboration around the data, around the research that you have gotten, and put a solid case for donor institutions and people who have money to tell them, look, we are indeed in deficit in those areas and those are the regions that we need to cover. Let us have a priority in presenting projects from those countries so that they can get funding.
I often see funding coming and going to people who live in the developed world and are engaged in the developed countries and universities. Which is okay, great. At the same time, it increases the gap often and results in more and more people, it multiplies to a digital divide in the academic circles.
One problem that arises often, the language gap. That's one of the predominantly biggest problems. What I believe is possible solution, it is to connect to the networks of Arabic academics. That's one suggestion that I'm throwing at you to see if you can react to be that, to see if there is a way to create a network between academics in developed countries with those in regions such as the Arab world. I would love to get your views on that.
>> MODERATOR: I have some thoughts on that. I'll ask that, networking, capacity developing, it is quite some of the ‑‑ some of my passion and involvement over the last few years. I'll hold on since I'm facilitating the discussion.
I'll ask, first, I think this question, it is very relevant in your context: Your great example of the effort that you have made working with young entrepreneurs, the students, coming out, giving them opportunities, going out there, seeking fundings, so tell us more about your experience in Mozambique. I told you, when you were a minister, were you using data really or were the decisions already made and you just ‑‑ I'll ask more about that too!
>> VENANCIO MASSINGUE: I want to ask you this question: You see, he's a man from promise, when he promises, he asks. I also promise you to answer.
>> MODERATOR: Thank you
>> VENANCIO MASSINGUE: The first part of your question, I would not agree more in what you had said.
I think ‑‑ you know, it is impressive for me when I see this graphic here. To paint Mozambique with the very dark red, to say that 100%, I don't know, what is it? It is not for you to explain to me. This should be self-explained, not only for me, but for the locals over there who are in dire need of using this information.
Maybe ‑‑ I'm sure now we'll become, you know, become friends, maybe we'll be good, we investigate the possibility of training initiatives, I don't know, maybe joining efforts because the minute the language, the culture, the habits, it also influences these numbers. Strengthening the local institutions to work with, I think this will be a very good contribution. I'm just provoking my new friend here. We have been discussing, sometimes I run away, I'm minister, now I'm back, so it is a ‑‑ it is a very serious thing, the issue, and what was being said, is that my model of development, it is rely on the youth, on the young people.
If you look at the statistics, you see the youth have the majority in Africa, in developing countries, and it is a very high pressure to give them opportunity to get the University degree, but if they don't create their own visions, they will wait to be employed, then you create a new problem for the continent. So training them to become entrepreneurs, to be able to do the job like my friend is doing, the consultant, you create your own business, I think this is are very, very important thing for them. I failed a bit in this, I became 50 without my own business. One day I will start. I'm still young, yeah. Okay. I hope I will start.
I feel I'm not a good example for them. I always tell them, you know, do something by yourself. This is a good field. This is very green. You don't have ‑‑ you don't have the fight ‑‑ you can use a lot of technology that they handle easily to collect data and to make the information available.
Yes. I think we have to promote more of this partnership to see the linkages between ‑‑ for instance, I have different heads now. Different heads now. I have the University ‑‑ different heads. I think we need more of such linkages with the other institutions.
I promised to answer the question about the statistic. I will give you a trick. It is a very important trick
>> MODERATOR: Yes
>> VENANCIO MASSINGUE: Make sure the graphic that's been shown, you take the five bullets and give it to the minister, don't give all of the graphics, just some of the bullets. We have overloaded with a lot of government information so you may have a perception, you know, that at this level people don't want to use, but it is because it is too much information, if you open one page, of introduction, okay, I'll read this later on, and then, of course, it became a problem. If you have a card, you know, something very simple, a small one that gives you the summary, the relevant of the topic that must be tackled, definitely you will see this being consumed at a very high level.
On the other hand, this is a trick I want to give you, very few ministers sit down themselves. They have an assistant. So we have to establish a strategy that allows the assistant to get that information and to make it yours. Sometimes you may have the most updated information, very well‑qualified, if my strategic international disaster doesn't read that, you don't get him to read it, it will not move on, even if they are really good. He or she will use what she gets access. I think it must be a strategy to see how can we make this information floating at the levels that allow the decision makers to make a decision. This is valid at the Heads of State level, valid at the minister level, it is valid. The director, the assistants, those are the ones that should be more and more ‑‑ may provoke ‑‑ had where it is written if a director asks you something, you copy that, you give it to the director. That's a good strategy. It will be produced, but, you know, et cetera, I promise to answer. Okay.
>> MODERATOR: You did. Thank you.
Reaction? Go ahead.
>> YACINE KHELLADI: Very quick, a question on trust then: Trusting, and when you trust the source, you trust the process. It is a decision making, they'll take what you give, what you bring on the table, more and more easily.
The other thing, it is about capacity building. The reality is when you train people on the ground in developing countries, they reach a certain level of efficiency, quality, things, immediately the international organisation big NGOs, other things, other people will take them and they'll move to the north. So the problem is how ‑‑ it is not enough to train capacity ‑‑ to build capacity, to train people to do research on the ground, you have also to provide somehow ‑‑ I don't know, the institutional, economical, whatever conditions needed to stay there. They don't just move to Geneva, New York, Washington, the next thing.
>> MODERATOR: Alison
>> ALISON GILLWALD: Thank you very much for that.
I was going to say we can provide a good case study for an example you were looking for in Mozambique where we worked with our partner for ten years just after them leaving University. I think it is about tailoring the data to the particular needs.
Basically we produce the initial analysis and research, take it into communication, academia. We test the rigger. We get the critique at that level. We take that essentially academic paper and we turn it into a policy paper. This can be anything between 25 and 60 pages depending what it is. That policy paper gets sent to the strategic advisor, researcher, whoever it is. What you take from that to a 2 to 4‑page policy brief which just highlights all elements of the study we want, we turn it into a number of policy briefs. We found that to be effective.
We use those policy briefs also for the media. Just to go back to the importance of the media, and what will often happen in some countries is that the minister will call and say I read in the media that this, this, why haven't you ‑‑ you say, well, we sent you a pre‑issue, you know, before we send it to the media, it is on your desk and you haven't bothered to read it. Your advisor. It are ‑‑ they inform each other, people go back, the researchers, they come back, they say where is that fabulous data that produced that policy brief or academic paper and can we use it? Again, the main people who are using that public data, they fund it from public funds, it is universities in the north, Harvard, ivy league university there, well‑known universities in Europe that are in a position to pick up the databases and use them which, of course, is wonderful, they open them up.
A big part of this researching project, which is why you were wanting to speak, the mandate was to build research which was uneven, and I think it is extraordinary, the value in Indigenous knowledge, local intellectuals and researchers
And the final point I want to make, it is that we work with ‑‑ our networks, our tuning, it not only has to be with universities of the north and south. In fact, our network started out as exactly that, as a university. As a network based in the London School of Economics, and then in Africa, Asia. What happened was, the African, Asia, Latin American network funded by IRDC, they began to share their research, sharing their methodology and produced the book you saw, which was an outcome of those researches. Industry or triggering that research.
I said that was the last point, one small point, taking that research, vigor, whatever, then taking the next step to creating Indigenous policy intellectuals. People that can actually push the boundaries of research, of policy agendas, that engage in an informed way in public processes against stakeholder, operators, to provide alternative viewpoints and to try to build a ‑‑ you know, a debate amongst the policy intellectuals.
You know, regulators in the U.S., FCC, they have contending intellectuals making their own policy more vigorous and informed. We're lucky if we can get sort of, you know, half a person and a dog to a public hearing. It is about really that evolution, that intellectual illusion and claiming that space as Indigenous and local intellectuals. Not to say we cut ourselves off from the north, anything else, of course not, but to engage in a more even and ‑‑ yeah.
>> MODERATOR: Stefano, and then Robert.
Any other questions by the way? No.
>> STEFANO De SABBATA: In terms of sharing, using the data, I think ‑‑ first of all, there is one I think important issue with content especially from the global ‑‑ this is a very good example I think.
Most of traditions, their culture, in many places, even local culture in Europe, for instance, in my region, it is on oral tradition, which is a problem with the Internet. Most of the time you want to refer to the publications, books. This gap is a problem when you don't have the references. This kind of problem has a big impact on the Internet where all cultures, for instance, are very much underrepresented. They don't have ‑‑ this has very big impact. We're ‑‑ we're getting to know more and more and more, and to know all of the other traditions.
Going back to the sharing of the data: I think that there are a few technical things that can help in remixing the data, for instance, taking care of versioning. The World Bank doesn't use any version control. They update the ‑‑
>> MODERATOR: What do you mean by ‑‑
>> STEFANO De SABBATA: If you go now, you download the Internet penetration rates from the World Bank, they're not the same ones that they have for ‑‑ if you download now the data for 2011 they're not the same data that I download two years ago about 2011. They update their information, but they don't say in anyway, anywhere, you don't know ‑‑ they don't have information on the country now, you don't know if it is available. There is no way to reference data in this way. Another problem, for instance, very simple things, using codes which makes life much, much easier when you have to mix the different sets, the web index, it has the name of the country but not a code, if I want to merge that information with other information I have, I have to manually move the codes and then match the information which can be problematic and take hours away. If you multiply that for all researchers around the world that have to do this job, it is a waste of time
>> MODERATOR: It is good that you said that now. We're trying to recreate the data. I would love to follow‑up on that conversation in a few months. I would love to.
Robert.
>> ROBERT GUERRA: I wanted to comment with what Alison mentioned,
To frame it differently, there is a need of an evolution of champions on the issues. A lot of times, whether it is for Internet openness, other issue, champions that emerge that rally the community, whether it is policy expertise, then they mobilize the conversation around the topic. Others, sometimes it is easy, it requires a lot of other skills for individuals to have. Sometimes a big challenge, it gets going back, sometimes it is if you're the director of the center, many others too, if you have a very good report, a publication ‑‑ we have had some that appeared in the major media, we didn't want to appear in the media. We want others to. The amount of time, energy, effort, it takes you away from the research, trying to balance who can represent the report is a big challenge.
I wanted to get ‑‑ finish touching on something that was mentioned for the younger researchers, newer researchers, sometimes it is a challenge breaking into the existing communities, finding opportunities. Coming to a meeting like this, maybe people here will meet experts that are here and be able to do that. I just don't ‑‑ I think it would be better to document what these communities are working on, that they don't stay between walled gardens, and if not done in these ways, you can easily communicate with them saying I'm working this time in Yemen, if you could quickly see in these issues there is someone else in another part of Africa, you may be inclined to either write them or to find others that can introduce you, that threshold to create that network sometimes is because of a donor. You should be ‑‑ other easier entrees to connect researchers from different regions
>> MODERATOR: Thank you.
Any other questions? No. No. We're running out of time.
I wanted to ask the last question.
I want to ask a last question, we're running out of time. I think it is important to discuss it before we leave.
It is ‑‑ you brought it up, Robert, the issue of funding. My experience with research, ICT Africa, even now with the Web Index, it is not necessarily easy to attract or get donors' interest in this type of work, in the collecting on the data on the openness, developing policies, et cetera, analyzing the data to inform the policy making. It is not an easy space. What are your thoughts, opinions on that? Any models you have thought of?
Perhaps a sentence, two sentences each.
Go ahead
>> ROBERT GUERRA: It was that issue that created this. This is because of the digital divide back when, and I think it is finding a way ‑‑ a ways to, one, to do that, and two, there is more time, energy spent that the conference talking about support for the IGF itself than perhaps for a lot of the different work. So it would have been nice, there is a recommendation going forward, there should be a donor session talking about these challenges in the next meeting. Then people could actually talk about them.
I mentioned earlier, the media takes a lot of time, energy, effort away from research. I don't want you to know what grant writing, getting donors, what it takes away from researchers as well. The private sector, others, they know this is something that others may be able to do, the duration of funding. Some donors are good for multiyears and others, the funding cycles are so short that basically you are really doing what the donor wants you to do as opposed to the research that's there.
>> MODERATOR: Filling in the frame
>> ROBERT GUERRA: It is an unresolved challenge. I'm not an economist or others ‑‑ I think definitely there should be more attention in spaces like this in terms of how the ecosystem of the number of companies involved, it is a multimillion dollar space for a small amount of donors supporting the research. Something is wrong there, and there should be other ideas. I won't propose them, but there are ‑‑ there should be more in this space. There should be far more funding.
>> MODERATOR: Yes?
>> ALISON GILLWALD: A sentence on this is very difficult.
I really ‑‑ I think that there is a dissertation in the political economy of funding. I think very much it is reduced ‑‑ they don't see the implications of the political agendas, of the impact of the commercialization and the modification of the data often in the name of self‑sufficiency, those things, they're really, really serious. Again, probably saying something that's unpopular, folks are trying to get funds, work together, feel warm and fuzzy together, but I think that the effects of the global recession together with the coincidence of conservative governments in many of the countries of the north and the drying up of traditional funding that came from a long development research trajectory has had severe implications for this sector.
In many countries in the north ‑‑ in Africa there is little state funding to do research. Some of the bigger economies like South Africa that have very, very big research consoles, those kinds of things, they also have political dimensions to that. It is opposed to the independent university research, it is very important too. The funding cuts from traditionally progressive donor agencies not driving their own donor agenda, not trying to get donor aid back to the country of origin. It has had a very severe implication. There is an enormous cutback in funding of work in ICT for the broadest context, policy work, the work that we're doing.
Also in universities in the north that have traditionally received funding, you know, aid, to do the development work, gender work, et cetera, it is cut back entirely. Those agencies that have traditionally funded, American aid agencies that are funded, big universities in the U.S., they're finding instead massive profitable multimillion dollar industry associations to go out and do honestly very questionable research, questionable quality of research that essentially has a market agenda, market research, opening up new markets in the developing world and because of the patrons that they have, the Secretary of State, whatever it is, actually they're the definitive reference point for our multilateral agencies. People are suddenly speaking about a woman on the web report by some big international company called a global report with 5, 6 countries actually examined and a total sample of a couple of thousand, 6,000 I think it was, and this is the data because there is no other competing data. I think the impact of the recession, the conservative cutbacks that have come because of the conservative governments in traditionally progressive, supportive countries, it has had a severe impact.
It raises the question with the kind of research, the reference points it has, with poor evidence, poor information, it is in fact better than no information at all. Are we better off or not worst off because of the poor quality of evidence of policymaker, what they're getting
>> MODERATOR: Thank you for that.
We'll continue the discussion over a drink because we didn't finish here. I wanted to thank you for coming, for accepting the invitation. I appreciate it. Thank you for joining the session.
I think this is the last session of the IGF. Please, go, enjoy Istanbul.
Thank you.
***
This is the output of the real‑time captioning taken during the IGF 2014 Istanbul, Turkey, meetings. Although it is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription errors. It is posted as an aid to understanding the proceedings at the session, but should not be treated as an authoritative record.
***