Sign New Mexico Arbitration Agreement Mobile

Document type sign arbitration agreement new mexico mobile

all right okay um hello so welcome everybody to our penultimate research exchange of the spring semester my name is Yvette subramani and I'm at citrus and we had two talks today was the first day by Quan shin and hidden the second be back upon parikh a couple quick announcements first welcome to our web viewers as you know we web cast us out to the other citrus campuses and thank you of course to infineon our sponsors for the lunches that you're currently enjoying and also today at three o'clock we are having our poster and judging session for the big ideas competition we're really excited about that because we've 13 great proposals by students they're going to be at three o'clock have the posters set up and it'll be in the atrium of our news citrus headquarters over sitar Judy hall and so if you're planning to go which I encourage you all to go there are still construction fences around there so what you really need to do is approach the building from the front through past you know up by the north gate past naval architecture kind of walk up to the building and you'll kind of will have someone there station to squeeze you through the construction fences and guide you up to the building but it is open it's just still under construction technically so today we are allowing people in for that this event so if you can George today at three that would be great and with that I would like to introduce quanshan who's going to speak to us first and a let's give them a nice warm it well and then this thing on is the mic on awesome hi I'm a second year of grad student in computer science and thanks for the opportunity to talk to you today about enabling quality data collection on mobile phones so just a real quick agenda I want to give you a background on existing practices in data quality and then we'll go and take a look at what practices and developing region context is like and then i'll talk to you about uh sure the system that we've built to address some of the challenges that we have identified and uh sure does these things so motivate the talk today I don't need to say that beta quality is a really critical problem but I'll give an example so in in the public health area each year 57 million people die and less than well I should say about two-thirds of those deaths are not medically certified unlike here in the United States where you know every death is counted and we know why a person dies most of these deaths happening in rural and developing regions don't get counted and so the metrics on things like say infant mortality aren't just aren't there and its really tough to do planning resource allocation and interventions when you don't have that kind of data and so there's starting to be these verbal autopsy programs where we where they send community health workers into the field and these health workers will basically carry a big stack of paper and for every person that died is fill in a 20 page form in the field to record why that person dies and they bring this data back to headquarters and the doctors or some machine learning algorithm tries to figure out why exactly this person died you can see from that this is challenging and that data quality is very important on the other hand data entry the process of capturing information at the source is really the best in the first opportunity to address data quality and so at entry time we have this opportunity to correct an error if we are able to detect it exists there's a lot of existing practice in data quality in my field database management systems we look at data quality in terms of data cleaning and data cleaning is often thought of as a post collection activity so once the data is in the database we run methods like multivariate outlier detection to find potential errors but the problem there is that once the data is in the database you no longer have that opportunity to correct it or rather it may be expensive or time-consuming to find the original source of the data to correct that information in the area of form design just talking with the wonderful folks here at the Berkeley Survey Research Center and learning about survey methodology I realized that there is a deep science to designing data entry forms it's it was surprising to me at first because I thought well you open up Microsoft Word and you start you know typing a name in a field and start collecting data but you know the science of designing these forms cover everything from how you want to think about wording of questions to the ordering of questions to the grouping of questions and within the question you want to specify constraints range constraints warning constraints and then between questions there's cross-questioned validation concerns as well after data has been put into a form we want to validate that somehow and the golden standard is is data entry so is is double entry with double entry it's the practice of just taking a paper form and you fill out everything twice by two separate people and then you compare any discrepancies go to a supervisor for arbitration and this is just this is the flat-out standard practice because it works really really well but you can imagine this is a this is a costly practice so resource let's see data collection and low resource organizations they in these organizations the ones that I've worked with they lack the expertise to to have that like they lack the form design principles they just don't know about them you know it's a this is a this is a science and they you know they because the barrier to entry for designing forms is so low they just sometimes don't think to include these practices as well in order to know for example on a forum about hiv/aids you know whether a field what are correct values for a field like viral load you have to have domain expertise and sometimes when the designers at designing is formed their lacks the domain expertise so basically they're lacks some expertise as well these organizations they run on a shoestring budget and they lack resources and so you know what ends up happening is well you need to collect some data well I have a list of fields and have a list of data entry widgets like combos drop downs and such I'm just map the field to the widget and and then I'm done so it's a very ad hoc practice sometimes it's done well because the designer has a good intuition but it's often not done in a principled way they because double entry is the standard a lot of these organizations well enact double entry programs but sometimes they're not not done very well but bottom line is that these double Andruw programs take the data and they ship them off to some data entry location and that data doesn't come back until say three weeks later and by that time even if it comes back to operations it may be too late to help operations so there's a latency problem as well with with double entry with the pervasiveness of cellular phones you know a lot of organizations are looking to mobile to solve the problems yet one problem is this double transcription where you would take data put it on paper take paper double enter into the database well what if you could directly transcribe that into digital form on a mobile phone that's a great advantage there are also new challenges on when you're collecting data on mobile phones namely you can't do double entry if you're out there in the field directly transcribing a data into a digital form right so some of the traditional practices are displaced there haven't been very many studies on data quality on mobile phones and the one that i found was in the recent I CTD 2009 conference and they're finding was basically that there's multiple ways to enter data on mobile phones the one that we all think of at least the one that I think of is when I just punch information into the phone on like a custom form application they compare that to what if they just use the phone feature of the phone and call an operator and have the operator type in the value when they did that study and found that using the phone as a mini form entry device actually results in ten times worse worse data quality so this is a problem before I go on I want to show you an example form that we worked with this is from the Tanzania national hiv/aids monitoring program the form you see in front of you is the patient registration form just to put you into context and so to summarize from the introduction what what are our challenges you know what do we want to do can we maybe supplement the lack of expertise some how can we take the practices from data cleaning and ply them at the time of entry somehow and can we reduce the cost of double entry so to address these challenges we we built uh sure and we took a data-driven approach and that is we learn from the existing data that exam we take that form take that form and the existing data is input we learn a probabilistic model over that data and we use that model to do several things everything you see on the slide on the right I'll go into detail so you know the resolution isn't that great but we use that model to one optimize the form design process to give it a certain goal to we use it to provide dynamic and contextual feedback while the users filling in answers and three we use the model to reconfirm questions that might be wrong but we reconfirm it in a way that we try to identify the questions that are most likely to be wrong and weary ask those questions first hopefully lowering the cost so building this probabilistic model at rushers core we represent the form fields of a the questions of a form as a Bayesian network and what we want to do is to learn the structure of this network so we want to learn the relationships between questions and the goal is that we want to give this model input values of questions I've already answered and have it give back output values of what's the confidence what's expectation and what is the error a potential error in in the input out what's the potential error in the input and what's the probability of the answers of the rest of the remaining answers so what we do is we first let the designer specify his or her constraints because the designer may have constraints like your name and address should always go first that we respect that we capture those types of constraints question groupings and then around those constraints respecting those constraints we run this structural learning algorithm to order the form questions in a particular way that benefits the later benefit the later dynamic feedback of phases let me take a brief detour to talk about our data sets so we use two data sets one is a survey of opinion an opinion survey done in the Bay Area it has about a thousand records and fifteen questions the other is a patient registration form the one that you saw from Tanzania we there was about sixteen hundred items data items and about nine questions so in order to prepare the data set for for our evaluation we removed any free text fields because it's it's difficult to learn from free text fields and difficult to predict free text fields as well like comments we took any continuous values like age or num basically we took numbers we discretize them on fixed interval ranges because the the implementation of the model we have expects this discrete values and then we treated absent data so the absent data is a big problem we treat absent data as a separate value to be predicted and we divided our data into a training set and a test set in the canonical way so the outcome of the structural learning that we do on these two data set you can see here for the patient data set it was able to learn from the data that the region code is correlated with the district code you can see that these these are arrows with directionality but ignore the directionality for a second this is basically saying that region code and district code are related and that's good to know for when we want to estimate the conditional probabilities of answers you can see on the right it captures a more of an intuitive idea of what we would expect questions in a survey to be related for example it was able to cluster together our opinions about spending on welfare spending on urban problem spending on the military infinium crime saying that one opinion will affect the other interestingly it correlates spending on military to your political party and your political party to your political ideology and so on so you can see it does capture the intuitive relationships between these questions so we continue to build our probabilistic model step one we captured the structure step two what we want to do is to fill this model with the baseline beliefs about what the distributions are so let's say you have a question as gender and we want to fill it with a baseline belief of what's the proportion of males to females we just go to the training set and we learn oh maybe there's sixty percent females and forty percent males and we fill the probability table in this question with those numbers now if we learn that the question is related to another question like for example age and age was discretized into ranges of say 0 to 10 10 or 20 20 or 30 then what we want to do is to count the proportion given 10 to 20 how many male's the females males or females are from ages 10 to 20 so we build up this for every single variable in every single relationship we build up this conditional probability table so that later we can use it we can say okay I see you've said your 45 years old and you're from this district given that fact what's the likelihood are you female or male we can sort of make that of prediction so at at this point we've built our model and what we can do is we can perform inference to get the distribution of unanswered questions given answered questions okay so this is the system we have the probabilistic model at the core I'm form field and prior data comes in to form the model and then there we've implemented a web interface so anything that runs a web browser including phones can run this front end which is just a simple form collection user interface the UI captures values from the user it passes up to the probabilistic model as observed values the model will pass back confidence scores and error scores back into the UI and the you i would use those scores for for some purpose shall cover next so before entry what do we do we covered this a little bit I just want to go through a little bit more detail we collect the designers specifications or constraints about what questions has to go where first of all this is very important because you know you would expect some questions to be in a certain place or expect some questions to be grouped together we collect those as as invariance and we respect those and then any anything else we run through this ordering Optimus ordering algorithm to front-load the amount of information that we capture the intuition here is that if we are able to front-load the information that we capture the front load that amount of information that we capture then we have higher predictive ability for the rest of the questions so the objective function that we use is based on entropy and the idea is that a question like the very top there the referred from question it's fairly uniform as a lot of different choices the entropy and that question is high so we choose that to be first okay and then we'll take two related questions district code and region code we know that they're related so if so we picked district code because it also has many choices and it has a fairly uniform distribution but once having picked district code we know a lot about region code we can fairly confidently predict what the region code is going to be so we've put that last because the information gain is not as great now if the designer expects region coned district code to be together of course we can put them together but if there's that room is that flexibility we allow this objective function to take over to maximize the amount of information we gain up front so that we can do these two things we can more reliably predict the remaining fields and also that predictive ability can be used to craft to customize the the user interaction I have this picture of of the man sitting on the curb and it's a practice that I earned from the folks at the survey research center called curb stoning and back when they I guess they still do door door surveys some devious surveyors would go to the house knock on the door ask a few questions maybe the high entropy questions and then leave sit on the curb and we fill out the rest of the form because they already know enough about that that person that interviewee to fill out the rest of the form i'm not saying this happens but you know this would our system would be good at prevent this kind of thing so we we ran a simulation to evaluate the question ordering with the hypothesis that ushers data-driven ordering can ask the most uncertain questions first and improve ability to predict the rest of the questions and the results were fairly positive you can see with the original form ordering in green and the random ordering in black a little bit more about the form on the y-axis this is the number of fields that we're able to predict correctly on the x-axis this is where we stopped so if we pretend like the user stopped at after five questions with the original orderings we're able to predict around maybe forty to fifty percent of the answers correctly with the usher information based orderings we're able to get that up to seventy or eighty percent so during entry there's a couple more things that we can do with the problem with the sort of the predictive ability one thing is we can actually dynamically change the ordering of questions on the user before the ordering that I talked about was static we determine the ordering for the form and it would stay fixed now this is the unpopular idea to dynamically shift the ordering of questions for the user but there are certain contexts where this would be extremely useful for example if you have an extremely large form and it happens to be a sparse form a sparse form being a form where each time you only fill out four or five of say a hundred questions okay now if you had a set of anchor questions that would anchor anchor you in to the set of relevant questions we can very quickly with dynamic reordering pull up those relevant questions for that user the other thing that we can do is to give dynamic feedback to the user to the user interface so the idea is that the user has entered in a few few questions and conditioning on those questions we can do these things we can use the probabilities to assess the quality of an answer that was given so if you see the drop-down box when the user selects the value we can color the drop-down value as say red well of course we'll have to train the user as to what red means but we can give them some sort of a feedback the good thing about assessing is that it doesn't bias a user it's a feedback that comes after selecting the choice of the question a more forceful approach is to nudge the user towards expected values and if you see this drop-down box here or rather this type of head box that autocompletes usually type type of head boxes when they auto complete the autocomplete alphabetically i'll show you alphabetical choices but what if we just switch that up a little bit and say well we'll make the first will order those choices by the likelihood ok so this changes a little bit of the physical metrics and the cognitive load for the user for them to find more common answers ok even more obvious way to come to try to guide the users toward more likely answers is to well guide or bias the users by directly showing them probabilities this is controversial as well this is obviously biasing them towards the norm and maybe you'll miss on anonymous anonymous answers or intent here is to show you the space of possibilities that you could of how you could adjust the user interface given the predictive power of Usher as a next step we're engaged in going to the field working with partners actually validate how the set of feedback mechanisms the i guess the costs and benefits of the set of feedback the set of feedback mechanisms okay so lastly after entry after the user has filled in all of the questions in a form what we want to do is we want to select out we want to rank order the questions based on how much error could there be in each question and what we do is we augment the Beijing network that we had before where it was just a network of these F values F representing being the random variable that represents each question with several other variables and nodes to capture the error probability I won't go into that and in depth but basically I'll say what we're able to do after we augment this model into what we call our error model we're able to run inference and calculate the probability of our which is error given a bunch of DS which is the observed value for other questions okay so this is loud this allows us to re ask questions and let's see how we did so uh sure question re asking basically we thought okay we can react when react some more important questions more quickly thereby reducing the cost and what we did was we took our two data sets and we randomly injected errors into the responses the graph that you're looking at here is looking at five percent errors okay and then we asked ok how many questions do we have to re ask and reconfirm in order to get to a point where we have perfect answers for all entries the random ordering which I would compare to double entry says well if Yuri asked all of the questions eventually you'll get to one that's seems obvious you have a linear gain in the correctness but with usher in within asked asking the first three or four questions you can see that we get up towards eighty to ninety percent accuracy so that means eighty or ninety percent of the forms within three or four questions have all the errors fixed the shows that we can reduce the cost of double entry to conclude we with usher we demonstrated that the automated probabilistic approaches can be used to design data entry forms and to promote higher data quality the system addresses each step of the data collection process from form design to filling to validation and our next step in the future is to validate this with our partners in the field so i have i don't know if i have time for questions I'm sorry okay absolutely I'm curious about the discrepancy between the operator assisted I'm reporting and the cellphone entered one do you have an analysis of that d what's happening why the operator what actually looked into that so in that paper the discrepancy in my opinion they didn't explain that such but in my opinion the discrepancy comes from the fact that when they did that experiment they had the user call the operator and say the answer is bananas an operator would say okay you said the answer is banana is that right and you would say yes and that essentially becomes a dynamic feedback mechanism in you know because it's human to human everything from the intonation to the length of the confirmation I mean it becomes a sort of a contextualized feedback mechanism you can compare it to either somewhat Usher could do or you can compare to what double entry is doing and that I think is where that discrepancy comes from all right you ready for a we got a question here let me get you it's only a comment but in the bad old days that we had computer cards to input to computers it was a real serious code we had to type it twice and then subtract one deck from the other deck as you're doing here days are long gone all right so uh ready I guess it took yes it upon pre-kick is going to talk about mobile phones and agriculture some of the work that he's doing with people in California so I will switch over we'll have a brief intermission don't leave your seats talk amongst yourselves youth that she's going to work for me like a right and a spur I right okay thank you very much so I'm going to talk about some of the work we're doing in agriculture specifically using mobile phones and other kinds of information technologies to improve the productivity and profitability of small farmers all over the world including here in the Central Valley in California so first one of the important things to recognize is most of the poor people in the world still make their living in one way or another from agricultural activities and as was demonstrated eter of that of that volume improving the productivity and profitability of these small farmers is the main way for many of the poorest countries in the world to achieve development to to progress along the economic development pathway now beyond that the other point I'd like to make is that if we can achieve increased agricultural productivity at with reduced natural resource consumption in terms of water chemicals damage the environment that allows us to achieve the same level of economic development at reduced environmental costs so in short economic development is not a zero-sum game you can have development with reduced environmental impact and so these are real challenges that face us as we look at how to achieve sustainable economic development and if we can do so with these constraints so here's how agriculture still works in many parts of the world this is India still labor-intensive small plots of land limited infrastructure and and and it's still very much a hands-on very much a hands-on process so a lot of the work thus far that looks at how improved information and information technologies can impact upon smallholder agriculture has focused on the provision of prices market prices so that farmers can best aside which markets to take their goods to now while this work is very very important the argument I'd like to make is that this really ignores some very important aspect of how agriculture works in developing countries and how markets work in developing countries so for example a lot of market transactions and decisions in the market are not strictly based on issues of price there are long-standing relationships between buyers and sellers that are really based on trust the buyer knows that they can expect a certain level of quality a certain reliability certain quantity from certain farmers and because of that that buyer is willing to invest in that relationship for example providing transportation providing advice providing guidance providing training providing input etc now these this really addresses a number of issues that also determine kind of price and relationships in the market things like quality productivity having appropriate transportation being able to manage supply chains traceability etc etc now achieving all of these goals which are important for increasing the profitability of small farmers a lot of these require the effective functioning of institutions that support these farmers most notably agricultural cooperatives and other forms of organizations that are better serving the interests of these small farmers now today I'm going to talk about two projects that we're working on that address some of these other information and knowledge gaps faced by small farmers in terms of accessing markets and improving their quality and productivity the first project is called digital ICS it's essentially an internal control and management system for achieving various kinds of certifications for a small farmer cooperatives and the second system about low is essentially to facilitate knowledge access and knowledge sharing between farmers and between farmers and experts so i should say first of all these products are both led by excellent students Yael Schwartzman leaves the digital ICS project she's based in Mexico and she's really conceived of implemented and deployed this system and Neil Patel leads the about lo project and that's really his his baby and so I should recognize that I'm going to be talking about students work as professors are often want to do first of all internal control so certification various kind of certification organic certification Fairtrade certification shade-grown tree friendly all these different kinds of certifications allow essentially a agricultural producers to obtain a premium above the the prevailing market prices based on the ability to label their products and based on the willingness of consumers that may pay more for such labeled products now for large farms this is fairly easy for them to ensure that the various practices that are required for fur let's say for example organic are followed on their farm it's all one can you know one continuous a several contiguous land land land parcels and they they manage the labor they manage all the practices and so it's very much a consolidated operation on the other hand in the case of cooperatives with many many thousands of small farmers that are all working together and in marketing and and supplying products together maintaining these certifications is strictly harder because you have to make sure each of those farmers is following those practices that that allow you to retain that certification and so what cooperatives do is they into the Institute internal documentation and and and monitoring workflows that allow them to ensure that each farmer is actually following the rules so to speak now this process of internal control is both data and labor intensive people need to go and visit every farm collect a pre-specified amount of information and that information needs to be processed and acted upon for example the head agricultural scientist for the co-operative needs to give targeted advice and feedback to certain farmers and also decisions need to be made about which farmers get to stay in the cooperative versus which may or may not be may be sanctioned in various ways so we've essentially built the system so the first thing is that you know this requires a lot of manual paperwork right now and what we've done in terms of you know some building on some of the ideas a cuong mentioned in terms of you the utility of mobile phones and doing some of this data capture in the field we built a system that allows essentially farm inspectors to capture inspection and survey data using a mobile phone while they're at the farm parcel directly so this includes things like the growing practices the status of the parcel the equipment what substances are used in cultivation etc etc and all of this data is captured by a mobile phone and now we're not only capturing structured data but we're also capturing images and audio so for example on this particular form there's some garbage on that farm which is a violation of organic certification requirements so that violation can be documented and can be acted upon appropriately we also capture audio comments from the farmers themselves in terms of issues they're facing in the transition between conventional and organic farming or new initiatives or new directions they'd like to see the co-operative take so this really improves the bi-directional communication between the co-operative and its members so the co-operative essentially a member-based organization the governance of that organization is highly dependent on the vesting of its members in its in its authority and so the members really have to be convinced that the co-operative is doing the best it can for them so having this kind of bi-directional communications ensures that the co-operative is responsive to the needs and the interests and the problems that are faced by the farmers that constitute its membership so now once this data is all collected we aggregated in a database that sits at the co-operative head office and now this database can be reviewed by the by the management of the co-operative to decide what actions need to be taken so for certain farmers they need might need specific advice and specific guidance and they can target extension activities such that that advice and guidance is provided other farmers may actually need to be sanctioned or even expelled if they've if they violated the certification year after year that really can't be allowed to continue because if even if an external certifying agency comes to that cooperative usually what they do is they sample a subset of the farmers to make sure that they're up to mark if even one of those is found to be in violation the whole cooperative loses its certification and loses whatever premium those are earning because of organic or because of shade grown so if that happened i would be a huge blow to the co-operative and so if it's found that a farmer is continually violating these requirements you have to make sure that that they that they rectify that so finally the system also automatically generates the reports that like I said is used for internal extension activities to decide what advice what guidance individual farmers need they also automatically generate reports for the various certifying agencies so certification for organic is different for importing countries like Europe vs japan vs United States each has their own paperwork to fill out it has their own requirement and so this system now automatically generates the different reports that are required for the different certifying agencies so what what's the impact of all of this so what we've done we've taken initially what was a very highly manual process that's required paper forms for doing the inspection paper forms for the evaluation and reports that were generated using Excel or another kind of database application now we've automated we've actually could capture digital data directly in the field that data is automatically transferred to a web application and all of the reports or automatically generated so we've we've there's a couple of the improvements that the system has brought to the operations first of all in the initial system none of the individual inspection reports were ever digitized and so if the co-operative wanted to know the history of a particular producer where they you know how they progress from their initial stages to where they are now that required going through files and files and aggregating all these reports and then making some sense of it now these inspector of these producer level reports can be automatically generated and it's very easy to get a bird's eye view of how this farmer is progressed from year-on-year the other thing is that this process actually is quite a bit more efficient than the initial system that required a lot of labor in both collecting data and processing it so what we found is that we've deployed the system with about nine hundred farmers for about the last six months and be in comparison to a control group we found that the the automated system reduces the time in the field by approximately thirty eight percent which is a significant improvement considering that most of the people's time goes and just walking from farm to farm I mean these are really remote distant farms that require quite quite a bit of time to get to so reducing that time thirty-eight percent was quite significant even more than that the processing time the time it takes to review all of these records and to make effective decisions has been reduced by almost seventy percent and if you say if you take these two labor savings which are currently you know you require fairly skilled people to do both of these tasks who are essentially paid on either an hourly or daily basis now if you reduce these labor costs it saves the co-operative approximately ten thousand dollars per year and that's approximately half of the inspection cost yearly inspection cost so the overall inspection cost around twenty thousand dollars now we've got that down ten thousand now more than that as I said we have performer records that can be used to track the progress of a farmer throughout a year and the feedback from the farmer is the kind of comments and questions and issues that they raised through the audio feedback mechanism can be used to better target the cooperative activities both in terms of its of its usage of its of its resources and in terms of the extension activities that it's conducting so based on these results the first partner we've had is actually signed a contract to adopt this system and to pay so my student yell has started a small startup company down in Mexico City and step go our partner in this project has essentially signed a service contract to pay her a yearly fee to essentially maintain and support this software and we've got lots of interest from other cooperatives who would also want to use the system now the other thing we want to do is that this cooperative now is it's a coffee cooperative located in Mexico this cooperative probably has the best data about its producers and produce the production process of probably any cooperative in Latin America and maybe the world and so having this better data does that impact its ability to earn a premium on its products so for example are there institutional or individual buyers of coffee who would look at this data and because they have greater assurances better ability to communicate with the farmers would they be willing to pay more for that coffee because of those facilities that's a question we don't know the answer to we're not sure whether or not that's the case but a hypothetical example is you could be at cold coffee down in rockridge and you could be having your cup of coffee that comes from Mexico and you can you know send a voice message to the farmer saying hey Jose man this coffee is awesome I I love this coffee and I just have never had coffee like this before and Hosea could be you know kicking it down in Oaxaca and be like damn man tap likes my coffee you know my my hard work you know all that work I put out into that farm that work is paid off and then that might sound kind of funny but it's actually something we've we've we've observed firsthand when we talk to farmers I mean they really put a lot of work into their coffee quality is very important to them and they'd like to be able to get some feedback that that hard work and that effort they put in an investment they put into that quality is actually appreciated by people who are consuming that coffee and even more than that if we're willing to kick in an extra nickel or a dollar to pay for that that'd be even better for them so it's interesting how we think about fair trade fair trade gives us transparency we know how much the farmer is paid we know what benefits they're getting from us buying this coffee but the other direction after there is no transparency the farmer knows nothing about as a fair trade consumer what I feel from consuming that coffee how much I enjoy the quality or the feeling that I get and so we really trying to build bi-directional transparency in coffee markets by creating communications in both directions so that's that story we're looking at a number of research questions around this project that that are motivated by this project looking at how including issues related to data quality but more generally how we develop usable forms on mobile phones that improve efficiency and accuracy how we manage all of this qualitative feedback and how it can be analyzed process and acted upon the third question is actually quite important now in terms of this software we've been able to make it locally successful in this particular context and as I said there's a lot of demand from other cooperatives would want to use the same piece of software on the other hand we don't have the same kind of training and support network that a company like for example Microsoft would have to support its software in all odd corners of the world so the hypothesis or the idea that we have is if we have this software essentially as an open-source piece of software would it be possible to build a distributed service network to essentially support this piece of software and now what are the kinds of incentives that are required to build such a network and finally we're looking at a set of empirical questions about how such software and how such systems impact the co-operative impact farmers and how that data could could make a difference in terms of the performance of this kind of coffee in markets so that's a summary of that first project so the second question I'd like to get into is a different question it's the question that farmers have so farmers have lots of different questions that come up when they're doing their work so they might have a specific kind of disease or a pests that's inflicted there their farm how do they treat that what how much and what fertilizer to you how much and what kinds of chemicals to use so the traditional way this has been addressed is through what's called Agricultural Extension and so that means either governments or universities or nonprofit organization send people to the field to advise farmers on what the best thing to do is so this is really usually pretty expensive it's not on demand you don't have that person there when you have the question it's not always you get the right expert the farm at the right time the person who knows the answer to your question might not be the person that comments for example you're you're having a problem with your chickens you know they're all they're all you know getting swine flu or whatever and they're coughing and sneezing and you got a spinach expert on your farm that spinach experts not going to help you and so that spinach expert will come by and say oh you got a problem with chickens you know let me call the chicken guy and a month and a half the chicken guy comes and the chickens are all dead that's the time you have the spinach problem and so so in general you know the these kinds of approaches are both expensive and they're not on demand and not likely to address the questions that farmers have when they have them so a system we built is essentially a voice based system a telephone operator system that allows farmers to call up this system access certain important announcements that are generally important leave questions and essentially listen to other people's questions and the answers they've received to them so how the system works is you call this system you navigate like a phone tree like we navigate when we call customer service or customer support we record a question the other experts farmers can call the same interface and they get a list of questions for which they might have answers you can provide answers to this system if I've left a question and I call back I'll get a list of answers to that people have left for my question and what so this is essentially a way of building something like a mailing list or discussion forum using voice as the medium so how can we bootstrap the same kind of knowledge sharing experience sharing question-and-answer processes that we do on the internet using voice and phones as the medium now the other thing that's done is the most popular questions the questions that most people have and the answers that they found most useful based on their feedback those are then packaged and rebroadcast on the radio so that more people can benefit from them so you get this way of making the radio information continually updated continually refreshed and based on the needs that people have and the advice and guidance that they found useful so here's an example of a farmer calling the system an expert possibly providing an answer maybe another farmer could also provide an answer and finally you take the most important and useful questions and answers and you make it into a radio show that people can listen to when when they have time so we've deployed the system again since January we've only given access to 50 farmers each of them in a separate village this is in india and gujarat and we found that that you know only with these 50 users and that's basically because of the scalability of the system we haven't put the amount of server as an infrastructure behind it to support more users but even with just 50 phone numbers that are given access we're getting about 3500 calls per month into the system with questions and answers and so that's more than two calls per phone number per day so it's clearly being used and what what we're seeing more of is that if one person has access to the phone and is calling the system the other people in the village are like hey listen man you know you've got access to that about low system now I want in you know I got this question that no one knows the answer to can you record my question for me and so that they that's how they get into the system even if they're not directly subscribed and the other thing we found is that you know now that the system is out there now that more people have the ability to provide answers not only so-called quote-unquote experts many more people are providing answers so now in the new system previously were only experts from the NGO or for the university we're providing these answers now three times as many answers are being provided by other farmers as the quote/unquote experts and so the rule of the experts really changes in the system from being one of being the bottleneck of information to really being a moderator or doing quality control on the kind of quality of the answers people are receiving in which which which makes their time more effectively spent so we were actually adapting the system as you vet said now we're adapting the system for used in the Central Valley in California for essentially the same kind of information access but in this case by ethnic minority farmers so farmers from the Hmong communities from the Chinese communities from the Spanish communities so that if they have information you know the extension services here they're not really able to access because all over the information is in English and all the expertise is in English so now they're able to pose questions in their own language have them translated by bilingual people and essentially have answers go through the same pathway and Maxim has not yet been deployed but it's currently in the design stage so we're looking at number research questions in this domain first of all if you have this kind of information if you're doing things like search navigation of content hierarchies with users with limited education limited literacy how do you make the user interface best correspond to people's existing mental models and the way they think about their problems and the way they think about the things that they'd like to know about so for example search using Google search on the web it's a learned practice you learn how to form queries appropriately based on iterative lyra finding those queries now that becomes very different in the voice based where you have to support that same iterative learning but using a very different way of presenting information so you can't really quickly scan so you have to prove up provide ways to quickly scan information to be able to access it in the same user friendly ways that we've gotten used to on the web set the second set of questions are also again about how this these types of systems behave in the field and how they impact current patterns of knowledge diffusion in it for me sharing so our farmers more likely to adopt practices they hear about through such a system as opposed to a radio program where it's some kind of disembodied voice are they more likely to adopt systems that are practices that are recommended by other farmers as opposed to institutional experts what are the cost-benefit ramifications of this system as opposed to the traditional method of people going to the field and providing advice so the users head of the empirical questions we're interested in exploring so that's about it just as a summary a lot of work is not about providing handouts but it's about providing existing knowledge systems assisting organizations existing institutions the ability to help themselves by leveraging the capacity and resources they have at their disposal and a lot of this work is about actually empowering those institutions and those organizations that are already involved in furthering the interests and addressing the needs of small farmers finally what I'd like to propose is that a lot of the models we that are explored in these two projects can eventually be transferred to other domains so for example the digital ICS project I see is fundamentally about improving feedback loops improving learning processes within rural development organizations and the second project I see much more about being how to facilitate peer-to-peer knowledge sharing and experience sharing and both of these for example would be very relevant for a domain like health care where you still want the same kind of feedback loops and you want the same kind o local bedding and sharing of knowledge so that's my pitch thanks very much have a two questions from online says first while I understand the usefulness of using cell phones to elect data it's a bit slow in time consuming to enter data on a cell phone have you considered maybe using wireless keyboards that's a good question i mean it so that i think that increases some of the hardware cost of the system so another thing we've done besides just just using mobile phones into the data collection is actually we've coded the form such that we have more discreet options than open open entries and so rather than say hey what kind of chemical this person is using we have options such as is a chemical a chemical be chemical see and so that that gives you enough affordances on the mobile phone to do what you would otherwise do youth with a keyboard and the second question is also how much user testing has a system undergone and with what populations so both systems have undergone significant user testing with the exact populations that are currently using the system so all of this hasn't been iterative in participation with users and continually refined based on their feedback and based on those evaluations okay let's take some questions in this audience so if you have a menu of choices like you know chemical a chemical be what happens when somebody has a question that really is different from anything anybody's asked before does that work so that's why every question has an other field and that other fields we capture essentially in the order audio recording we don't do text entry yeah that's a good question though it's a great project I'm wondering one of the one of the the in terms of transferring the model to this context and I can see why you would use cell phones in india in the poorest areas and the farmers in this context in California we're ideally the infrastructure at least is going to be deployed it's that there's yeah planning on why not use the inter based system I'm thinking in terms of cost because I see that a lot of the ethnic populations that your would be dealing with are benefiting there they're using a lot of the cell phone functions that are could be done via the internet and I see that it support so basically they're paying more for less in that they're paying per minute per per minute yeah so it seems to me like the kinds of things that you would do with the internet you you know with that that kind of pricing do you see that as a barrier I don't know what the price system is in India in terms of the sofa yeah I mean that that's a good question and so so our experience with just having some discussions with ethnic minority farmers in the Central Valley is that most of them it's not a cost issue necessarily but they're just not comfortable using computers nor do they have the experience and so we're looking at the phone in terms of just being more accessible and comfortable because they have experience using it now the cost issues you know I think broadband internet costs similar to what I pay for my phone line at home I'm not sure what the underlying infrastructure alcocer of those two systems but I don't see a huge advantage in terms of cost of using the internet as opposed to using voice especially when if it's not something you use daily or I mean think for significant hours every day the the kind of benefit we're seeing of this is not necessarily comparison between internet and voice I think both could be effective carriers for the kinds of content information we're talking about given proper access devices and given proper infrastructure so we're pretty agnostic with regards to those alternatives but I think essentially disintermediated and making some of these services asynchronous allows us to more effectively use the resources of the experts and I think our argument is that this approach is cheaper than how the expertise is getting out whether or not use voice or the internet then the current ways there are kind of physical transport intensive give any other questions thanks very much