>> Good morning, everyone.  I'd like to encourage you to come up front because if there are not a lot of people, that way we can make it more inclusive and involve you in conversation and the OECD has brought some documents up here that you may like to have a copy of.

>> MODERATOR: Good morning, everyone.  I think we're ready to begin.  We don't want to take up your time.  I know everybody is busy and has other engagements.  Good morning, every.  My name is Barbara Wanner.  I am with the U.S. Council for International Business which served as one of the workshops 104, integrated policy framework, key to realize digital inclusion.

Another of our co‑organizers was the Internet Society, Jane Coffin, one of our speakers is with the Internet Society and the government of Colombia.  In this workshop, we will focus on sharing practical insights on the value of framework for digital transformation in fostering economic prosperity across all sectors and improving societal well‑being, inclusively.  This workshop will use the OECD's going digital policy framework.  There was a main session yesterday, or rather it was in the main meeting room that focused on different tools for realizing digital transformation and the OECD made a presentation about the Going Digital project.  This will look at the policy framework that was an outcome of that project.  It will consider it as reference and explore the values and barriers that may arise in integrating the framework and how it's Going Digital toolkit may help to overcome some of these challenges.

First, I'd like to acknowledge Lauren Crean from TMG legal who will serve as the all important substantive Rapporteur for this session producing the written report for all of prosperity.  I'd also like to acknowledge Bruno Martin Dosantos who has provided a lot of the behind the scenes work.

And now I'd like each of the speakers to introduce themselves and very briefly explain what they do to provide some context for their remarks so let's begin with Ben.

>> BEN WALLIS: Good morning.  My name is Ben Wallis.  I work for Microsoft.  I'm a regulatory policy analyst there based in Seattle and I work in, across intergovernance issues.  I spend a lot of time with the Internet Governance Forum, mart of a multistakeholder advisory group.

>> JANE COFFIN: Good morning.  My name is Jane Coffin.  I work at the intersection of policy in technology and development.  The teams I work with every day are out building internet infrastructure working with governments to shape policy and trying to include different open standards in the work that we do from the technical side.

>> ALEX COOKE: Good morning.  My name is Alex Cooke.  I'm the counselor for industry of innovation and science for the Australian government.  Which has been leading the work on the Going Digital project

>> DAVID GIERTON: Good morning.  I'm David Gierton.  I work at the OECD on the Going Digital policy for the just mentioned joint commission.  I'm going to talk about my work in detail in a minute.

>> MODERATOR: Great.  Let's move to David.  In undertaking the Going Digital policy, the OECD developed a framework that would provide policies but also provide practical advice concerning a country's status, if you will, in the process of digital transformation and some practical tools to help them overcome some of the barriers and challenges so why don't you walk us through the Going Digital toolkit so it's not so mysterious and we all understand how it can be accessed.  If you have access to the internet, you can access this toolkit.  Thank you, David.

>> DAVID GIERTON: Thank you, Barbara.  Before I actually come to the toolkit, I think it might make sense to give you a little bit of background on the Going Digital project that we carried out and are still carrying out at the OECD which brings together just about every policy community.  The project started in 2018.  We had a conference at the end of the first stage of the project in March 2019 and we are now in the second stage of the project.  I want to just mention two key outputs from the first phase and for that, I ask the technicians to switch to my screen, if that's possible.  The first is this publication which summarizes over 130 outputs from this first phase of the project and has a lot of policy analysis and insights on key policy issues and policy recommendations, structured on the basis of the integrated policy framework that we'll talk about a lot today and second publication that is focused on measurement structured also on the framework which provides the state of the art of measuring digital transformation and provides a roadmap on how to improve that measurement.

The third key output from the project is the Going Digital toolkit, which we have launched as well in March this year, s first version.  Just making this a bigger so I hope you see it better.  This should be better and as you can see there, what you see is basically the integrated policy framework.  It is also built on that framework.

Before coming to this actual toolkit and showing you a little what you can do with it, what governments and policy makers can do with it, I'll talk you through this framework.

So, we developed this framework because we saw that many governments in our member countries and partner countries are addressing digital transformation but they do it in a fairly ad hoc manner often still in many corners of government and not necessarily in a coherent and coordinated manner so the rationale for developing this framework was helping governments to improve coherence and coordination across all the policy areas that are affected by and are affecting digital transformation so the framework basically consists of these seven policy dimensions that you see here on the screen in the seven colors.  And each of these dimensions groups a number of specific policy areas that should be looked at jointly and are in need of coordination.

So, I'll walk you through each of those dimensions and give you a bit of an idea of the actual policy areas that are in each of the dimensions.

The first one, access, is about high quality access to communication networks and services, as well as access to data, which is becoming increasingly the foundation of the digital economy.

And to enhance access, some of the key policies that should be considered are communication infrastructures and services policies, regulation, obviously, for example, to put the technical enablers in place, competition policy that are crucial to lower prices and improve quality of communication services.

Investment policy, in particular, for investment and infrastructure so infrastructure keeps up with the growing demand for data through‑put but also regional development and policies, for example, that are very important to make sure that connectivity doesn't stop at the city border and sort of rural urban divides can be addressed and reused.

The second dimension is about the use, the effective use of digital technologies, by all key actors involved in digital society and economy so individuals, firms, and governments in particular, and to make sure effective use happens, key policies include digital government policies where we see one crucial aspect to go beyond e‑government which is sort of digitalizing additional services to adopting a more holistic and user‑driven approach, investment policies that enable firms to address not only no ICTs but also intangible assets crucial to making use of technologies.  SME policies, we see SMEs as a weak point, really, in the economy, in terms of taking up digital technologies and policies can play a very important role in helping SMEs to catch up and thrive in digital markets.

Business dynamism is key, too.  In other words, core policies to help everyone, have the skills that is needed to succeed both in life and at work and digital privacy, we see mistrust still as a real important barrier for all sort of actors, individuals, organizations, to adopt digital technologies and make proper use of them.

Coming to innovation.  Which is really fundamentally underpinning the transformation itself and to foster in the digital age, key policies that should be included, entrepreneur policies, for example, regulatory burdens are often still hindering start‑ups to innovate and experiment in digital environments.  Again, SME policies, for example, public policies can play an important role in helping to foster open innovation and open science.  Again, digital policy, but here, with the view on open government data that can play an important role and obviously a whole range of policy integrations that can very directly affect if and how new business models thrive and fail and business experiments just to highlight here, regulatory, regulatory sandbox that can play an important role in everyone's mind.

From digital transformation, for good reasons, in that both quality and quantity of jobs being affected by digital transformation positively and negatively.

So, to ensure that the transition to watch the digital world on which everyone can succeed happens.  Key policy include labor market and regulation that promote a successful and fair transition.  Education to empower people with the skills needed to succeed in digital work.  Social left behind including those in new forms of work that we see in digital markets.  Tax and benefit systems need to be fit for transforming labor markets as well as for new forms of work and are not always yet, and again, regional policies can play an important role in addressing regional imbalances in transforming labor markets.

Society is going digital, took and we need to make sure that this is inclusive and improves well‑being.  Key transparency to improve these objectives include social policy.  Obviously, again, tax and benefits can play an important role as I mentioned in the context of transforming work.  Education and training are key to reduce existing digital divides and to strengthen the foundational skills for life and work that often, we see still being fairly weak in view of what is needed in a digital environment, as well as lifelong learning that is crucial to company people in this ongoing changing world.  Other policies, like environmental policies and health policies are crucial as well.  Environmental example.  It's very important to unleash the challenges to tactical and global challenge and technologies can play an important role here.

Again, digital government is key in this aspect here to boost civic and stakeholder engagement, in particular, into the policy making process.

Coming to trust a fundamental transition for the digital economy and fundamental environment flourish, to strengthen, first of all, from our point of view ‑‑ sorry, there's something.  From our point of view is crucial to take digital management or risk management as a central approach for all trust related policies.  For example, digital security risk management is crucial to be integrated by all actors that are acting in digital environments ranging from individual organizations.  Privacy is key, too, here, obviously, national strategies are an example, an important tool to address privacy forgot only from a technical issue but from a whole of society perspective.

con perspective is coming to the fore very strongly again key to protect consumers in all digital environments, that includes new types of markets like peer to peer markets and also SMEs again, an important focus here for policy in that they often are the most vulnerable to digital threats and the least well equipped in dealing with digital risks.

Coming to market openers, digital technologies transform how firms compete trade and invest, leading to greater competition in some market but also tilting others to greater concentration and to foster market, key policies to consider are trade, we still see important trade barriers in many areas, investment, again, barriers to international investment.  Financial market to ensure good access to all firms for going digital.  Consequently, it is key here to less the dynamics.  We see ending taxation, to protect the purpose in the digital age.  There's a separate quite important project.  These tax implications of the digital economy and how to review and revise national and international frameworks for taxation.

So, all these seven dimensions could be taken strategy, so, in the publication I showed you on the Going Digital policy on shaping and improving lives, we have actually an additional chapter on helping governments in the development and implementation of such coherent and comprehensive strategy.

So, just one word about themes that you may see missing at this high level of these seven policy dimensions, they can't possibly cover everything.  There are a number of cross‑cutting themes that you probably realized I've mentioned a couple of times throughout different dimensions.  For example, different skills for the government but also inclusion and so, given that they are so cross‑cutting, they are not one single dimension by themselves to just go a little bit into inclusion, if you look at access, inclusion here is a crucial theme of the rule, closing the digital divides we see.  In use, it is essential in terms of closing the gaps in technology diffusion, for example, between small or large firms, but also, amongst low versus high educated individuals.  In innovation, it's there in terms of leveling the playing field and supporting young firms to succeed.  In jobs, it is obviously very prominent.  Think of social protection and skills, education and training policies in particular.  At the heart here of working towards more inclusion and society, that is more than obvious to create equal opportunities to participate in the digital society for everyone.

So, I would stop here, but, as you see, it basically goes to the whole framework and we did recognize some of these themes are explicitly here under the themes angle where you see that here's digital government, for example.  There's other themes, gender, for example, if you go on and explore here what is seen, that's a little heads up on what I'll show you in a second, you recognize these colors so there's actually indicators for different dimensions that are seen here because there's cross‑cutting themes, more particular, on policy analysis and cross‑policy guidance.

So, this brings me now, basically, to the rest, maybe not the rest, probably the current core of the toolkit.

Which is, again, as you see, structured around these seven dimensions of the integrated policy framework, access, innovation, et cetera.

And what we did here is, well, two things had that the toolkit delivers is.  First T helps countries self‑assess whether they stand in their own digital transformation.

And second, it provides them rich source of policy analysis and policy guidance that the OECD has developed over the years and keeps developing under each of these dimensions so it empowers policy makers in their own policy work and more generally speaking, provides them comprehensive one‑stop‑shop for the development of digital strategies.

So, to run you just through a few functions here, what you see here is basically this overview of what you call core Going Digital indicators that is really a selection of indicators.  There's over 200 indicators in the publication I showed you earlier on in measuring the digital transformation.  This is a selection of central indicators that were shown based on relevance, but obviously also based on data comparability, comparable data we have so we can actually get a pretty complete picture for most of the countries and compare a country to the OECD average, that's what you see here.  So, let me just select a country.  Let's see Belgium.  So, you see Belgium here.

Plighter circle and the OECD average, the black dots here, excuse me, the Belgium is the darker part of the spider chart and the OECD is the lighter part.  So, you see the difference between Belgium and the OECD for each of these indicators and gives you a really nice quick overview of where Belgium stands and then you can compare it, obviously, to other countries.  Say, Sweden, for example, and you see Sweden is actually performing really well on a range of dimensions, Berlin than average and actually better than Belgium, not everywhere.

And then you can look into more detail on each of these indicators.

So, if I go, for example, here, on people buying online, you get to the next level, you have all of the countries here represented as dots and then you have the full data with each of the countries in the data set and in addition get a lot of breakdowns so you can look at the individuals buying online by H.  Breakdowns here.  You can obviously select individual countries if you're particularly interested in the comparison of selected countries and you can instead of only looking at the latest data available, you can look at time series so here, you have over a decade of data that shows you the illusion, you can obviously download the data by charts so it's designed to be really useful and easy to use.

Maybe one last word about the toolkit, which is currently what you see here is a strong focus on the data side of things.  We have, as I mentioned, a second leg of the toolkit which is the policy side of things so here we show the most recent and relevant OECD publications on issues covered under each of the dimensions of the framework and the policy guidance in particular, OECD counsel recommendations.

And we're currently working on building further, both the data and the policy side of the toolkit but in particular the policy side, where we are looking at quite substantial list of policy issues that many countries are struggling with that keeps ministers up at night but where some countries have actually started with some approaches.  Compact notes, to keep them emerging, sort of as a very handy source of information to get an overview and insights on what other countries are doing on these issues that are really difficult at the moment.

So much for the toolkit.

>> BARBARA WANNER: Thank you so much.  I think that was very helpful and the think the OECD has done an extraordinary service to us all in terms of creating a tool that we can access so easily.  Because of the importance of this to our discussion, are there any discussions on the toolkit and the seven essential implements of the policy framework.  Please P.

>> Hi, everybody.  My name is Marguerita.  I represent educators because I've been working as a teacher in private education.  So, David, my question goes mainly to you.  You mentioned education skills quite a lot and it's crossing all the sectors, more or less.  So, we're speaking about digital transformation here.  There are a lot of policies happening here.  But there is one big issue in education that it never gets to educators and there is a vicious cycle that exists in education in curriculum design and this is not only coming from my experience as an educator, we have a huge community of educators in China all over the world but it is also backed up by this publication which was published recently that education is not updated.

There is no outreach that goes directly to people who actually create curriculums and they have no idea what's happening in the digital world, in the world in general, in terms of the future, what jobs are coming.  You mentioned the jobs that are going to be affected and teachers who directly focus on developing those curriculums, they don't know about that and that's a big issue.

You have a great toolkit that shows data and can help people in policies to understand what's going on and how to update policies, but what about direct implementation of those policies of the future in terms of hands on activities for people who are actually creating the next generation's education.  Thank you.

>> DAVID GIERTON: Yes.  Thanks very much for the question.  That is, obviously, skills are one key area covered here and as I've said, it's cross‑cutting.  Now, you're getting to the next level, so to speak, in more details.  And on that, you would go in any of those, well, let me go to the theme page, and to the skills section here.  One second.  So, what you have here, we talked about the data and if you look at the publications, you have a really nice entry point at the work the OECD is doing about skills.  There is a huge load of work.  The past years have focused very much on education in the digital world.  There's the last of 2019 skills outlook that is almost entirely about this top you can and the one issue that you allude to is to include the teachers into, or skilling the teachers as condition for having actual effective education in schools that includes digital technologies because the inclusion of digital technology is only effective if teachers know what to do with it and how to do things and to actual teach skills to pupils that are relevant to the digital world.

I just want to highlight here the OECD digital skills strategy which is maybe a nice starting point for you, to get at it, teacher education.  Obviously, teacher education is in itself a whole field that our education directorate is looking into.

I just want to mention just my country, because we are here, Germany, which just launched a huge digital ‑‑ spending 5 billion on equipping schools with digital tools.  Now, that's not the only thing.  They have done this based on a strategy that was developed two years ago by the states of Germany, which involves, which are doing the curricula of the states, and they have done a coordinated effort involving all the school representatives to come up with a strategy on education in the digital age that sort of creates a conceptual basis or the principles and guidelines for curricular development and this is now matched the digital schooler which brings in the technology.

So, you need these two sides, for sure.

>> Can I say something on this, please?  Briefly.  Absolutely.  It's a great example of Germany doing this implementation.  It would be amazing to read about it so that other countries can also use it as a framework but not all the countries are capable of doing that now so the main issue still is how do educators use those amazing research paper that's OECD publishes n because they just don't even have access.  So, there are examples like ambassadorships, teach as the G ambassadorship is a great way to spread out information about SDGs to educators.  Why not create something like this for educators so that more people can read your papers because from 2,000 members of my community, very few actually get to read papers.

And the work you do is worth reading and it prepares educators for the future if they have access to it and if they are on it.  Thank you very much.

>> BARBARA WANNER: Okay. I think we have some other insights from our speakers on the questions you raised.  Ben, what does Microsoft have to offer here?

>> BEN WALLIS: Thank you.  And thank you, margarita, for your question, very much.  Agree that it's really important to find that way to reach through to the teachers themselves.  Microsoft sees it as vital to adapt education and training to provide knowledge and skills that are going to be in demand for these new technology related jobs.

So, there were just a couple of examples that came to my mind that related specifically to your question.

We, these are both in the U.S.  But, we have a program that Microsoft runs called TEALS.  Technology, education, and literacy in schools.  That started out where engineers, which I'm not one, but engineers within our country started just volunteering to go into schools and teach computer science.

And then over time, the teachers are sitting in on the class and it helps them develop a curriculum in relation to computer science.

The other example is something that LinkedIn worked on in Los Angeles.  So, the city of Los Angeles, LinkedIn provided data to the city of Los Angeles to help build out information and on demand skills part a local government program designed to help grow the city's tech economy.

LinkedIn used the anonymized data of its employers and doing that were audible to map out the education history of the local tech industry and also understand hiring the skills most sought in the structure so as a direct result of feeding that data, I understand that five community colleges adjusted their curricula or their classes and more than 400 young Pex were put into kind of paid internships in the tech sector

>> BARBARA WANNER: Alex, do you have anything you'd like to offer?

>> ALEX COOKE: Thank you.  The question is really important.  We have an added layer because the schools are located in our territories rather than the structural level so we have another point and therefore another point of failure.

So the way it works is by setting out overarching policy and we about three or four years ago set up a digital curriculum as part of the national curriculum standards.  The way we've been trying to, I would say incentivize the digital economy in general is by providing skills and resources available program to provide STEM in particular to come into schools.  We often find that particularly in terms of the visibility of the digital economy, it's important to see what it ‑‑ looks like.  Because as you point out, teachers have a picture when they go through University of whatever that was the system would look like.

We recently released a system of supporting artificial intelligence in schools which is a component as part of our AI capability fund measure he and the intent is to develop curriculum development to research, educators and teachers to tell those students what those activities are like so we're trying to approach it through that way so lifelong learning through microcredentialing is another area that we'd be particularly looking at as a keyway of delivering that.  So.  Yeah.

>> I would just say I work at the grass roots level could quite often and we go to places.  We don't suggest we know everything about the cultural environment and every country will be different.  Obviously, even in countries, it will be different region to region within a country but I think you've hit on something important about that spirit of education and transferring the learning but it will take ambassadors of digital literacy and transition.  Both of my colleagues have mentioned that and I think of course with something like this framework, you're going to need to unpack it a little bit and figure out how to layer the information so that people can pass it on.

And educators are perfect for that type of thing and I think it's an integrated approach because Ministries of education working were Ministries of it communication, it's become almost a given now.  In the past, it wasn't, that people need to work together across the Ministries and government for an integrated approach because this is an integrated sectoral approach across different pieces of economy and society.  Thank you.

>> BARBARA WANNER: Thank you, speakers, and the gentleman in the back, you have a question?

>> Yeah, I work for the Canadian regulator of telecommunication services.  It's an amazing policy framework and toolkit.  I'm just curious.  You identify women, elderly and low income as some vulnerable communities.  Where do people with disabilities fit within that framework?  And is there a reason why they're not explicitly identified because arguably, they're just as vulnerable as those other three communities.  Thank you.

>> Yeah, I absolutely agree.  They are part of the framework.  I have not touched upon them.  There's many things I have not touched upon but if you go into the publications, I mentioned this toolkit and the publication from the first phase of Going Digital summarizes over 30 reports so there's a number of reports that touch on inclusion issues and goes more into details of vulnerable groups.

So, if you're interested, happy to follow up on that.

>> BARBARA WANNER: Okay. We'll move on here and maybe provide some country examples of how OECD countries and nonOECD countries are endeavoring to implement this framework.  Alex, the government of Australia played a very instrumental role throughout phase 1 in implementing the Going Digital project.  Has your country faced any challenges and how have you addressed them?

>> ALEX COOKE: Thank you.  In responding to this, I might be able to respond to your question as well in terms of the role of addressing people with disabilities.  So, I'll touch on that at some point.  So, late last year.  Australia released its stand alone policy for the digital economy after an extensive consultation period so if you heard David's presentation.  I wouldn't say lock step, but they were happening in parallel with each other.  Advertise the integrated policy framework was being p put forward by the OECD.

So, this had an advantage in that Australia was able to take into account the integrated policy framework as we were developing our policy, which is called Australian techs futures.  It focuses on four key areas which align quite neatly with the dimensions of the Going Digital toolkit, people, services, digital assets and the enabling environment.  Against the category of people, we classified developing Australia's skills and leaving no one behind so we'd situate under that the job skill society of promoting social prosperity and inclusion.

One of the areas and answer to your question which we had a specific focus on is looking at where we need to improve our activities.

Australia has a program called the Australian digital inclusion index and it looks at a range of particular areas where we need to focus on social inclusion so they include older Australians, women, indigenous Australians, which is another important area, particularly I imagine Canada has got an area of importance here, people with disabilities, people in low socioeconomic groups and people living in regional and remote areas.

So, again, the focus on regional and remote is not one every country would be focused on.  The role of the digital inclusion index is to provide us can with an information base so we can understand and monitor whether the performance of our policy and interventions is working and adjust accordingly.

Going back to the program, the policy, we have another category which is around services and how the government can better digital services which relates to the dimension of the integrated policy framework around use, where Australia has been trying to establish itself on the front of digital transformation agency.  Against digital assets, we have the dimension of building infrastructure and providing secure access to high quality data which, again, relates to the increasing use of digital technologies and data but also enhancing communication and data and finally, the enabling environment and under the enabling environment we would include cybersecurity, reviewing our regulatory systems and for a country like Australia, which is quite geographically remote, questions around market openness and access to international markets and ensuring that trade barriers are minimized and don't allow Australia.  The first time we've brought everything together under one banner, we've looked across the enterprise to find 20 different policies, our first attempt informed by what the OECD work was doing.  It's bringing everything under some common thematic headings.

Just finally want to be able to point to the really important part of work which is part of the ongoing efforts of the OECD which is part of the roadmap and this is where we try to bring together whether they're having an impact.

So, we, in Australia, have been following a parallel track here as well in trying to look at how we can better measure our own activities in the digital economy, and we're very focused on what the OECD is doing in its roadmap for measuring the digital transformation.  We all know there's a lot of areas of the digital economy not picked up so we feel like we have a pretty good handle on issues around handling trust and market openness and so on so we're particularly focused in looking at where those areas can be picked up in terms of both Australia's work and then going forward and the OECD activities.  Thanks.

>> BARBARA WANNER: Thanks very much, Alex.  I'd like to go back to this issue of enabling environment and having the important policy and regulatory environment to enable the digital economy to flower.  Ben Microsoft has been a global leader and integration and what are your thoughts on the integrated policy framework and importantly, how would you like to take this forward and create additional elements that would facilitate the development of new technologies and their applications.

>> BEN WALLIS: Thank you, Barbara.  So, I'll talk about why an integrated policy framework is important to business.  Particularly a global one, Microsoft.  But, first, I wanted to say a little bit about why we think the OECD in particular is such a good organization to work through to produce this toolkit.  I think it's noteworthy that the OECD is an organization that's focused on sustainable development and innovation.  It has an evidence‑based approach, it embraces a multistakeholder model and I think as David took us through those examples, with the graph chart, the OECD maybe uniquely has the data and the capabilities to try and pull this b kind of thing off.

So, Microsoft believes that in order to achieve the vision and the potential digital transformation, there needs to be integrated holistic joined up approach to policy programs.

And there need to be clear objectives set by governance at the national level, just like the ones that Alex has talked about in Australia.  And those national governments need to find way to convene agencies together on how to achieve the objectives because it's important to putting the forward is an important step, we do acknowledge the implementation and it has traditionally been important for governance.  Not just the variety of index across digital economy but also because it's not intuitively obvious always where the discussion needs to take place within national governments.

Not every government, but most governments, probably, don't have a Ministry devoted to the digital economy.  So, there isn't, necessarily, a natural convening platform within a government and I think that's why it's important to have an implementation toolkit in a practical way of how to implement a holistic approach to policy making.

>> BARBARA WANNER: In particular, Ben, and again, I'm thinking about the enabling policy going forward, Microsoft has proposed the use of regulatory sandboxes to enable experimentation and innovative uses of new technologies in a safe environment, shall we say, a safe, regulatory environment.  Can you just explain to people what this concept refers to and how you think it might be useful?

>> BEN WALLIS: Thank you, yes.  So, I'll take artificial intelligence as an example.  The potential benefits and concerns are generally agreed upon to the question is how you put guidance in place to encourage the development of AI technologies in a responsible way.  And how do you encourage the use of the technology in ways it helps meet the global challenges.

We also enable technologies to get the most use, there needs to be experimentation in the ways that technologies are used.  So, taking healthcare as an example, if you want to use AI in healthcare, there are clearly potential harms and we wanted to better understand those through expectations, through experimentation.  And you can also use experimentation to understand how to apply existing laws to these new technologies, so, no healthcare, that might be how privacy might apply when AI is used in healthcare.  Had.  Fear steep consequences of potentially violating an existing regulation.  Then a regulatory stand box is something that could help.

So, if we look at Singapore as an example, the government there published AI ethical guidelines and importantly, they followed up by providing examples of ways in which data could be shared in a responsible manner.  And they put out this vision of a collaborative data sharing platform.  They were clear that not everything is known.  They made it clear that if a company has an application they're not sure about, they could come to the government, apply for potential exemption or a regulatory sandbox to enable it to do the experimentation.

So, what's interesting, I think, about this.  It's not simply about making space of course private sector to innovate.  It also provides a way for governments to understand what the actual harms are that would require regulation.

Existing regulations sufficient to address those potential harms or is something new need indeed and is that something new simply a new interpretation of existing laws?  Or are entirely new regulations need indeed

So, I think regulatory sandbox is a really great mechanism to help answer these questions and they can provide a mechanism to enable innovation and experimentation, new technologies, and new business policy in a controlled manner while also helping governments identify the harms that need to be regulated against.

>> BARBARA WANNER: Thank you very much, Ben.  That was very useful.  I think, you know, we've heard this term regulatory sandboxes battered around this week and so I think it's important to understand in very practical terms what that would help and what it would enlighten us about.  Jane, the Internet Society has been part of the global society of internet users here at the IGF and I know you work at the grass roots with individuals who are in the infancy, perhaps in the infancy of developing their internet connectivity.

Please share with us how the Internet Society has endeavored to facilitate capacity building, what are you hearing at the grass roots and I guess importantly, do you think the OECD needs to provide more targeted guidance for nonmembers in terms of going digital.  Thank you.

>> Thank you, and I guess the overall answer is yes.  There are more targeted and probably very specific use case examples because I was just thinking audio visual because some companies that don't have access to this are going to need ways to unpack it and implement it.  But going back to your question.  One of the challenges of nonOECD members will be integrating and understanding the context as it relates to them at that industry level and understanding how countries are working, or not, together, as Ben had mentioned and Alex, you've got to have an integrated approach across different industries.  If you don't, that cross‑sectoral focus might be lost.  Some countries I've worked in, the Ministries were more like businesses and when countries are in transition, there may be silos and governments that prevent that integration and cross‑sectoral and that's really important that you need to recognize what you're going into.

Data gathering was mentioned obviously by David, Alex, and Ben you, I think touched on it earlier, too.  Evidence making for policy is something we all take for granted but some countries may not be at a point where they're able to gather that level of data in order to provide information about what they're doing and to level up progress that they've made.

This is not something we also realize in the work we do that we've got to go in and create an same, to help enable the environment, you've got to know what the environment is.

And if you don't know what the user base is, the level of connectivity, how well government works or doesn't with each other, you may be starting off with what you think is a great way forward and it's actually going to be flawed before you get halfway there.  We touched upon the issue of access and infrastructure, this tool will be great to use as a baseline but if you have access to infrastructure, the critical employment infrastructure is something you'll want to focus on first.  Access to the heart of something is what we do.  Anything that enables a country to change that environment is going to take human engineering.  We often all go in, we think the project is going to work, it's easy, but it's 80 to 90 percent human engineering.  You've got to get the local buy in, know who your stakeholders are and it takes layers of engagement.  Going in one time to talk about something really doesn't work.  You've got to come back.  And that's where I think through the OECD, stakeholders like ours both at grass roots and technical communities, you can create digital ambassadors across that level to help with training with local governments.

Another aspect that I was thinking of when David was speaking is how you would integrate this into an already complicated environment of digital transition.

Some countries are still going through the incumbent operator phase.  I dealt with this 20 years ago.  I'm thinking, gosh, all of that great policy work we did to liberalize, privatize, bring in more certification, back haul from the landing, things are being challenged right now at the infrastructure level due to security concerns at cybersecurity which I'm finding as someone that's trying to go in and build infrastructure, it's really hard when governments are all of a sudden clamping down on Social Security and infrastructure rollout in the name of cybersecurity.

So, you would need to go in and look at ways in stages and phases to talk about that transition and how they would implement the framework.  Providing use cases.  There are great ways that governments have learned.  I used to work for a government, so, I know.  And you can't, which Ben has said.  This is something which is experimental and sandboxes are amazing.  I've worked with governments that thought, they put in a dispute resolution, it was done.  They were fabulously perfect.  They're not perfect.  And if you go in with a mind‑set that this is it, you're not going to make progress.  So, I think these sandboxes are critical.  We're working with CTEL through the government of the United States with a deployment and regulatory sandbox.  I think the OECD could do well with partners and members to go in at that regional level to figure out how to integrate.

>> BARBARA WANNER: Thanks very much, Jane.  Do we have any questions for Jane?  I know Alex is prepared to talk about what Australia has been doing in the Pacific rim but I just thought first I would see if you have any specific questions for Jane?  Okay.

I know Australia has shared knowledge and helped capacity building in some of the small south Pacific islands for this connectivity probably is an issue

>> ALEX COOKE: Thank you, yes.  And this goes to the question of fact of going in and making sure you don't use the tools and as part of what's called a cybercooperation in the Pacific rim area.

We've been currently investing $37 million over seven years what we call our cybercooperation program which is part of championing open and secure and free in the secure cyberspace so the size of this program has been growing overtime as think we've been developing buy in both within our own government but also seeing success object ground.

Through this program, we work with a range of implementing partners on the Australian side, through our Attorney General's department, our federal police, multilateral organizations such as UNOEDC, Australian Pacific policy institute and academic institutions and we've been focusing predominantly on building cyberresilience across the endoPacific key.  Maybe something that's not been reflected as much.  I think this goes to the question of how you connect digital companies is the alignment to achieving the SDGs so we've been looking very much at connecting our eight activities around using technology to address the SDGs there.

We've been doing that through four particular dimensions.  So, one of those is through delivering training, including tailored training across the region to build understanding of what norms of responsible state behavior are.  Cyber bootcamp p, cyber law courses and a range of other activities.

We've also been funding activities to fight cybercrime.  Building cybersecurity capability in countries such as Myanmar and the Solomon Islands.  Also, where we've been providing blockchain technologies to particular countries so we're giving these countries access to cutting edge elements of the digital economy to try to bring them up as quickly in leap frog, some of the activities that developing countries have already got to.

Things like e‑governance readiness assessment, some workshops are also key parts of this.

Just finally, throughout the Pacific step up initiative Australia has been providing funding to provide what we call the corral C which is a submarine fiber optics cable which brings next generation connectivity to the people of Papua New guinea and canary islands, also connects to Norel and Tarel Island so these are key parts of what we would call access under the Going Digital framework but building that bridge under the OECD member states and out to other countries.

>> BARBARA WANNER: Excellent.  Ben mentioned artificial intelligence earlier and I'd like to go back to that topic.  Earlier this year, the OECD endorsed principles on artificial intelligence after probably a one year plus effort by a group of dedicated experts.  How might the integrated policy framework be utilized as a complement to help police department the OECD's principles and most importantly, ensure development of trustworthy AI.

We've been hearing a lot of this, that concern this week as well.  I invite any of the speakers to address this.

>> BEN WALLIS: I'm happy to start because Microsoft has more to say about AI as a developer and provider of AI technologies.  So, we think about this in, from two angles.  The AI ecosystem and society at large.  And so, Microsoft, as I said, we're involved in developing AI technologies but that's only one part of the AI ecosystem.  It also includes the people and the organizations that are involved in integrating, deploying, maintaining, and operating AI technology solutions.

And we think that you need to think about that, as soon as a whole.  From Microsoft's perspective, we think about how we can develop our technologies in a way that would increase awareness of the need for trustworthy and responsible AI throughout the ecosystem.  Not just what we ourselves do as a company.

And then the second element, second angle that I mentioned was society at large, what is needed to ensure that AI technologies are actually adopted and deployed to their utmost potential.  And there's a couple of examples of ways that Microsoft is trying to make a contribution in this way.  We have an AI for earth initiative.  That's a five year, 50 million‑dollar effort to put Microsoft's AI tools into the hands of researchers and scientist that are working solid environments, related to agriculture climate by diversity and water.  And there's a combination of money and technology.  We provide grants for projects.  We also make available open source tools, models, infrastructure, data, and APIs to support environmental scientist.

And the other example is, a program called AI for Accessibility.  Where we're focused on working with others to solve challenging problems for people with disabilities by making software smarter.  Making devices smarter.  And making them more contextually relevant for people with disabilities.

And for both of those angles, the AI ecosystem that we're part of and society as a whole, I think the integrated policy framework can give us guidance and that might be in the form of specific requirements to feed into the AI ecosystem or requirements like trust, skill.  Maybe touching on something that Jane said, it can also provide guidance about actions that can be taken in complementary policy areas so not necessarily how AI should be developed and how it should work but connectivity and access so that there is more democratized access to AI and the deployment of these technologies doesn't exacerbate the digital divide.

>> DAVID GIERTON: Yes, just maybe to add to that, that the way we look at any specific technology like AI or any other technology in what we would call the technology ecosystem that underpins digital transformation, any technology, really, can have effect across economies and thus could be relevant in any of the seven policy dimensions.  So, just to illustrate with AI, if you think of access.  For example, new business models that can be enabled by AI.  Jobs, obviously, implications of AI for the changing nature of jobs and the changing tasks that are carried out within jobs.  Society, the ethical implications, and one pillar I want to highlight where I think there's very strong similarities with the guidelines that are trust, there's a couple indications for trust, to highlight, for example, the third principle on transparency and explainability or the fourth on robustness and safety and also the fifth on accountability, all of these are really key to enhancing trust and that is a key for transformation.

And looking at it the other way around, what is in the trust dimension of the framework, thinking of security, for example, and privacy, these are also crucial areas where the implementation of the policies we discussed there help the development of that AI and sort of provide the context which AI can flourish.

>> BARBARA WANNER: Jane?

>> JANE COFFIN: I'd like to just rest on the point that David just made about trust.  It's that building trust with the community and from the governmental layer, from the OECD, from Internet Society, from Microsoft, there's some fear, obviously, now, with respect to technology and I'm not going to get into the why.  There's been a lot of discussion about this this week.  But one thing I think seven recognizing is that information is critical if you're building a community of trust and a community of interest so getting that data out from both the OECD side, the government side, and all of us who know that connectivity leads to socioeconomical development.  How we promote that is very critical right now to the future of how that infrastructure is developed so if we can help build that level of trust and understanding, you can definitely get through a lot of complicated issues and help with adoption of technology and that infrastructure.

>> BARBARA WANNER: Thank you, Jane, that leads to the next point I wanted to raise about trust which is an important pillar of the integrated policy framework and certainly a theme we've heard throughout the week.  How do we find that balance.  Developing policies that enable digital transformation.  It's a difficult balance, but I know some companies, some organizations, have devoted a lot of time to developing, enabling that innovative potential to be realized by also safe guarding, p I would be curious about this approach to this and the other speakers.

>> BEN WALLIS: Thank you.  Microsoft sees privacy and security being inexorably linked.  Against cyberattacks for cybercriminals and other malicious actors so protecting our customers and the wider responsibility, we take seriously and we believe privacy is fundamental as more of who we are and what we do is recorded and stored in a digital form, preserving this right to privacy becomes more important and reasonably difficult.

So, Microsoft as a company takes a principled approach to building trust and makes strong commitments to privacy, to security, and compliance.

So, looking first at what people talk p about privacy, and spend a lot of time and money to ensure not just that we would be in compliance to the data secured by Microsoft but that we can support our customers to ensure their compliance.

And I didn't just make sure this happened with regard to the data people in the European p union.  We extended the protections employed under the GDPR to all of our customers and there are a couple other principles in this area that we follow and would recommend to others and we think organizations should be required to establish sound privacy practices.  Privacy laws should require that organizations can demonstrate that they've established sound privacy policies and that at a minimum, they can ensure compliance with legal requirements.

And that principle should apply both to organizations that collect and process data, so you might think of a about bank or hospital, and to those that just process the data on behalf of other organizations, Cloud service providers like Microsoft.

The other point is about the way you design privacy laws and how that might or might not enable data analytics.  We wouldn't want privacy frameworks to be so restrictive that they prevent governments, businesses, and other organizations from using data and analyzing data to draw IP sites as long as that's done in anneal manner.

And one way that privacy frameworks can achieve this balance is by encouraging the deidentification of data sets which allows researchers to continue to innovate but not at the expense of the personal data of specific individuals.

>> BARBARA WANNER: Any other comments?  Alex?

>> ALEX COOKE: So, from an Australian government perspective, we see the integration of privacy and security being two frameworks as well.  So, we support an infrastructure that drives national growth and protects national security and promotes international stability.  We think this actually drives the foundation of trust and confidences for governments, businesses, academia and Civil Society to then share its ideas.  So, we recognize that governments have to play a certain royal in regulating the internet but we've been very clear that it's not one of control and that state centered models would restrict and inhibit the government and restrain the operation center inhibited by connectivity.

So, we look rat data localization for this perspective as potentially a barrier to trade and so the most prominent example of this is in the W it, O context at the moment so Australia has been looking forward to the authentication you have the moratorium on institutions like electronic which we see as a Big Bear year to electronic and digital trade.  We would like to see it made permanent which is in line with the commitments that we made in the number of FTAs that we've been negotiating.  Concluded in the negotiation.

The main reason we're against these is that it will increase the cost of goods purchased online and we think that means it will lower the possibility for economic prosperity and act as a disincentive for customers to engage in the digital economy.

The other aspect mentioned around data localization is how to manage privacy.  So, Australia recognizes the importance of effective privacy protections that avoid barriers to information flows and to ensure n continued trade and economic growth.  We have a privacy act which currently allows the cross‑border disclosures of personal information in a range of specific circumstances to facilitate free flow of information across national borders while ensuring that the privacy of individuals is respected.

So, we have a privacy principle which requires that before an entity discloses personal information oversee seas, it must take reasonable steps to ensure that the recipient is not breached the privacy principles in Australia.

So, we've been working towards data flows.  To facilitate the flow of, in that context to review with a privacy act in March this year.  And a process to set up legislation to strengthen penalties and enforcement under our existing privacy aspect.

So, we think it's actually on the penalties and enforcement side rather than the regulatory side, per se, that we can achieve some of these outcomes.

The reforms will include a binding online privacy code which will apply to social media platforms and other online platforms that trade in personal information which would allow them to be transparent about the activities they take and to make best practice standards when taking consent to use personal information.

Just finally going to the point made by Ben is that we are very interested in understanding what the technology called developments are that might address some of these issues, Cloud‑based services, things like the, with privacy regimes while maintaining security.

>> JANE COFFIN: I would be remiss from an institutional perspective at the Internet Society if I didn't say that we are promoting strong and end encryption as away to also secure information and build more trust as well.  I think it's important, many countries in certain regions may not even have focused on privacy.  So, the examples that my colleagues have given are a way that to take advantage of other governments and other organizations implementing at a very simple level how to deal with privacy, and from a governmental it and corporate perspective.  We've worked closely with the African union to just, to come up with a, some cybersecurity principles but also touch upon privacy.  These principles with r worked in a bottom‑up fashion but I think you still, once you have principles, you can't just rest on those principles.  You've got to go out and work with people from the bottom up perspective to understand what they mean so they can implement them.  I think that's something I've seen when I was a regulation and policy guideline maker, if people don't understand how to implement it and make it walk, it's going to sit on a shelf and may not be effective.

So, that's something, I think, that can be done through the OECD, through companies, through governments, through organizations like ours, to really understand what it means, to put those regulations policies and sort of cross sectoral guidelines together.

>> BARBARA WANNER: This is a very important and sort of tricky issue, striking a balance.  So, does anyone have any questions from the audience on any of these points that they'd like to raise with speakers at this time?

Okay. I'd like to turn quickly to an issue that Alex raised, data localization and we understand how Australia views that.  But, I would be curious for the other speakers' views on its efficacy.  I know USCIB as an institution does not support data localization.  We feel that it is indeed a trade barrier.  And that it actually makes a country that has these sort of policies more vulnerable to privacy and security breaches because the data is centralized in one location but I'd welcome other panelist's views.

>> BEN WALLIS: If I can kick this one off, I think we've recognized there's distrust and there's a concern about the power that increasingly resized with the large technology firms and I think you can see these data localization laws where countries are requiring data to be kept within the country's borders.  I think you can see those digital policy solutions as an effort to counter act that balance, reflecting in this larger trend of distrust.  And those data localization laws can be well intentioned.  They can also be costly to implement.  You lose the efficiency that comes with the global scale of the Cloud.  You undermine the fundamental benefits of Cloud computing.

This is an interesting element that I don't know if people have picked up on it.  It's possible for companies like Microsoft to build Cloud unless systems which we do for the U.S. government and the Chinese government but the extra cost and complications make it harder for smaller companies to do so so kind of creating a barrier to entry for local Cloud providers.

We think that a more effective approach is to adopt regulation that's in line with global standards or global contract standards that protects personal data regardless of its location.

And such an approach can help to improve resilience and security, and to make data processing services more efficient by reducing latency.

But, of course, I think it should go without saying that it's incumbent on data processing companies like Microsoft to understand the laws in each country of origin and make sure that data is managed accordingly.

>> BARBARA WANNER: Thank you.  We're kind of down the home stretch here.  I notice we're down to like our last 15 minutes or so.  Just to kind of wrap up the importance of having healthy public/private partnerships and collaboration has also been a recurring theme I've heard throughout the week and it also appears in many OECD recommendations such as the AI principles, the 2013  OECD privacy guidelines and the 2015OECD security guidelines.

Alex, I understand that the government of Australia has proposed building relationships between digital end users and government so‑called co‑creation.  Would you elaborate on that proposal?

>> ALEX COOKE: Thank you.  So, the discussion around co‑creation arises predominantly within two areas.  One is the slightly more positive one and one is the more challenging one around terrorism and vital extremism content.  I wanted to touch briefly on the development of our AI ethics framework which has been following the model partly informed by the work they've been doing on AI to develop ethical guidelines for trustworthy AI.

And Australia has been in the process of consulting on the development of those and is now going through a piloting process of those guidelines.  So, as an example where we're trying to set up a framework that isn't enforceable, per se, to allow companies to understand how they could apply it in this new area, and AI is a fast moving area.  We have only just established AI principles several months ago so trying to make sure that he with don't stifle innovation and create a regulatory framework that we're trapped with, allowing our companies in Australia to actively engage and give us feedback on that work.

And so, we've been working with the Commonwealth bank of Australia, the national Australia bank, Telstra, which is our big telecommunications company, Microsoft and Flamingo AI piloting these principles and we're also working with the AE as observers O‑their group to understand the lessons they're learning with their process which concludes in December.

So, this is one example where governments are seeking to develop regulation, but to do it in a way that is openly consultative initiative.

The other more challenging one which is around addressing terrorist and violent extremist consult has arisen as a result of the Christchurch massacre.  There with a some discussion this week about some work that's been going on there.  There's two developments I wanted to point to.

One is around the GIFCT.  Some people may know, the global internet forum on counterterrorism.

It's an example of a forum on online platforms coming together and trying to take responsibility, let's say, for their role in the dissemination of terrorist and violent extremist content and to put into place practice that's will stop that from happening, take down rules and very basic things, making sure that companies understand the definitions of what terrorist and violent extremist content actually is.  This is particularly important for smaller companies.  In our experience, we understand that the larger companies can deploy quite a large number of resources.  I think Facebook said they had 350 people focused on this problem in their organization.  Smaller start‑ups, SMEs, don't have that capability.  There are some roles SMEs need to play in brokering that activity.

Taking the government arrangements which will have governments participating in that.  The other strand of work is some activity that's being undertaken through the OECD as establishing what's called a voluntary transparency reporting protocol which is an example of what we're trying to understand what the common definitions of terrorist and violent extremist content are and trying to incentivize companies.  The creation idea is that the first workshop which took place in Boston a month or so ago brought online platforms together with governments from a range of different areas from our cybersecurity, counter terrorism, digital economy areas, had a very robust, frank, and private conversation.  And that was the most important thing.

It was very much a process of learning lessons and I think this is an area where there's a lot of opportunity for governments to work with online platforms in what's moving very quickly and whether we can extend that into places that we can explore more closely.

>> DAVID GIERTON: Which I found really fascinating.  Colombia just, we just undertook a review of digital transformation in Colombia and they're hard working on the implementations of this review.

And one of the initiatives they've just recently announced is to work together with Corcera, the online digital learning environment and with the objective to have 150,000 students being trained in programming and other ICT skills by mid‑2020. ‑‑ sorry, 2022.  Which, I think is a really nice example for people but also an excellent example for how to use digital technologies to address a key challenge in this case, skills shortages in Colombia by using digital technologies and to tap into resources that are not necessarily there in the country, but that can, in a very efficient manner, help build up the capacity in the country.

>> JANE COFFIN: I'll just say that without partnerships we can't do anything, so, it's public/private partnerships that help anything.  We've worked recently and I'll give you a specific example with the government of Georgia to put in a local infrastructure in high mountain villages.  Without the government of Georgia, our local chapter, the internet service provider association, Tushedi economic foundation, this is the region, and other aid, we'd never be able to do what we can do.  To design partners, partners, partners, other community networks that are being built came together and everyone is putting in either human resources in experience or funding.  So, we've actually created that environment of collaborative connectivity development but we're also trying not to be the single point of failure in that collaborative partnership where we bring in other partners, so, we start to pull out as far as funding goes and they come in to help create that more sustainable baseline of the pyramid but you can't build infrastructure and you can't have some of all these fabulous things if you don't have that connectivity so I think the public/private partnerships as David and Alex were saying across the different sectors and at different layers with content you need to put in place, especially with the Christchurch call.  You've got to work together.  There's no way to do it in a siloed manner anymore.

>> BARBARA WANNER: Any other questions or comments from the audience?  Yes, please, sir.

>> I'm from Afghanistan telecommunication regulation authority.  And my profession is basically charter content.  It's good similarities that I work for the Telecom regulator but by profession, I'm a chart accountant.  The problem we have in our country is the financial inclusion which paved the way.  The digital inclusion of the internet which paved the way to facilitate financial inclusion.  In Afghanistan T there is a big challenge for us.  Accessibility wouldn't we a problem.  We can make the plan how to access, and how to bring the people to access to the internet but the main problem is how we should tackle the problems in order, so that the people can rely on the internet which paved the way and make it much easier to familiarize with the financial inclusion.

The big problem we have is the reliability of the internet transactions and that is the Big Bear year that we couldn't succeed.  We work together, for Afghanistan and the regularity of the Telecom sector, they work together but still, we have the same problems.

Any experience or anyhow we can tackle those problems?  Thank you very much.

>> JANE COFFIN: I would say that bringing people to your country or working with you to bring some of your experts to those experts to help create a better understanding of what you could do with both from a security perspective and that financial inclusion.  It's complicated.  And it takes time it and expertise.  You do have in the country.  I've known some people from your regulatory bodies and your Ministries but it is a matter of expertise and sharing that expertise and knowledge with you and what works best in your country because you know your country better than anyone.  But the question is, what is it on that infrastructure side, the banking side.  GDPR, for some layers, not only at that area in my life.  Providing that financial stability for the country as well and build up the reserves and trust in stability.

But, I would say this is a matter of working with your experts and international experts to come together to figure out a solution for you because a lot of training would be involved and that's where a lot of governments and countries do have that sort of expertise than knowledge transfer and you could teach them a thing or two, I'm sure, as well, going forward but it's a matter of trial and error as well.  And we can't afford that when it comes to of the banking system so also something we've been promoting on the encryption side is that banks really have to have strong encryption.  People forget you need an IT team.  But, it's not just any IT team.  You need an experienced IT team.  That could be a partnership arrangement with some organization to help you.

>> BARBARA WANNER: Okay. Everyone, I would like to thank all of our speakers for generously sharing their insights and expertise with us and I'll hope you'll join me in giving them a round of applause

(applause)

Thank you, everyone.  I really appreciate you participating and I'm sure any of the speakers would welcome your questions following the meeting.