About it the right way. We are proud to have passed our peer review. And finally, as it relates to our audit recommendation follow up program, what we realize and know is that the value of the cfas work is not just in the recommendations themselves but the corrective actions implemented by the City Departments. Its ongoing basis at 6, 12, 18 and 24 months from audit issuance, resulting in the recommendations issued by us from july 1st, 2010, through 2017 and 98 of the recommendations issued from july 1st, 2010, through june 30th, 2015, is just a testament to the work of the department and the seriousness they take from the audit work performed by our team. So of the 190 recommendations not implemented, which is only 37 or 1. 7 , they are more than twoyearsold. Generally because either its a major implementation or there needs to be some type of bargaining unit discussion before anything can occur. So thats the end of my presentation. Congratulations on passing the audit and thank you and your staff for the monumental amount of work that you do to keep the city on track. Thank you very much. I have a few questions i want to ask. Start with some of the deeper material you have in the report, i agree, excellent overview of whats going on. In terms of the nonprofit monitoring, you give us numbers of contractors who had findings which was a very low figure. Can you tell us something about the dollar value of the contracts opposed to the fact that there was 21 contractors with one or more findings . What was the dollar value of the contracts . In order to be in the pool subject to citywide monitor, you need to have 250 million or more and contracts with more than one city agency. There are a lot of nonprofits that meet the threshold, the findings are not specific to any one contract, theyre for the nonprofit as a whole. Its a little harder to answer your question in sense of dollar value issue in any one of the findings. Say we have at least one of the organizations on the list is i think arts and Cultural Center that owns a building and administers Community Programs in the building, maybe does Community Arts grants or Something Like that. Theres a range of dollar values being worked by that particular agency like owning and managing the building to doing Community Grants so i dont know if that answers your question. Is there a list that lists the contractors yes. Our report that was issued from the program lists theres a couple details in the back where you can see by name all of the nonprofits that are monitored and the ones where they have multiple findings, theyre named and then the few where we have what we call elevated concern status, which is high level of monitoring and specific action plan thats required of that agency to bring it into compliance, i think we had five agencies that we made that status designation for this fiscal year. Also another way of looking at this is what areas of services do the nonprofit contractors provide. For example, are you finding a greater compliance for those who provide Human Services versus Administrative Services or any kind of way of looking that tells us more . These are all nonprofits, theyre all in different types of public and community service, health, substance abuse, social services, arts and culture, they kind of span the whole service side, not bricks and mortar, construction or anything like that. I would say that the issue is often nonprofit that doesnt have the infrastructure it needs to deliver services properly. Financials, making sure they do proper tax filings and a number of other compliance issues and then Board Members where being on a nonprofit is a lot of work, its difficult to get people to serve on the boards and to function properly as an Audit Committee and review the financials, keep the staffing to a high level of compliance. A fair amount of technical coaching is helping get financials structured, beefing up staff or hiring outside staff if they need to and trying to get stronger people on the board capable of oversight functions and diversify fundraising and that kind of work. I know nonprofits are under a lot of pressure these days, so im i think whatever youre able to learn from this and share with them is only going to benefit them and the city. A question on the whistle blower work you all do, i was surprised to learn the other day that the whistle blower law only applies to employment, and not retaliation for example in services. Say you have a city contractor who is providing assisted housing and someone in the assisted housing files a complaint that money is being skimmed or people are jumping ahead on the list or what have you, in retaliation theyre ehe evicted. Thats apparently not covered in the law from what i understand. Is that your understanding as well . Correct. As it relates to retaliation, the side of the house we live on is the investigation side. We dont handle the retaliation. That is on the ethics side. And i know that theyre quite aware and are working on various pieces of legislation. Do you investigate non employment whistle blowing . If its around getting city funds, well look into the complaints. I believe a citizen can use the program to report abuse of funds by city employees. So so rigging a waiting list would be abusing the funds. Yes. I was looking at some of the audits you listed here and i didnt find all of them online. The one you did on ethics at the end of december. So as a just as a reminder, as i spoke regarding other assessments, that work is done under other assessments and the work belongs to the Ethics Commission, therefore it is not published on our web site. We have done the work and monitored it, and then we remitt it to Ethics Commission and they will then go through their process and determine where they will post it. But you have it listed as issued on 12 28. We did issue it. I was responsible for the work, monitored it and worked with the contractors but it wont be on my web site. Its not under my authority. Got it. Thank you. I have a real specific question, last Year Department of Supportive Housing and increased oversight, do you as part of your order look to see if theres good coordination among the agencies i guess they would be agencies or what, nonprofits who are providing homeless services. I have heard numbers of in excess of 200 million a year that the city spends with outside agencies and i know theres a lot of them and thats a lot of money. I wonder if anyone is working hard to see there is good coordination and isnt overlap of services. I think from a broader perspective with all of the work that has been done with the various agencies that are involved in providing homeless services, i believe this is why we have department of homelessness and ensuring were using our dollars in the most effective and efficient way as going on currently today from what i understand and i can see. And then as we move forward in our audit planning, we can determine are we doing a compliance audit in that area or an audit that looks at operations and that hasnt been determined for this fiscal year but next fiscal year we can consider that. Okay. At this moment in time if we went to look at it in that way, they really havent had enough time with the new re right, right. To move forward in a way that i think would be beneficial from an audit perspective. Perhaps in the next fiscal year. Yes, we will consider. Thank you. Thanks. No more any Public Comment on the presentations . Good morning. The 2015 16 civil grand jury issued a report on the Whistle Blower Program and the most recent change in the ordinance restricted the scope of protection under the ordinance as well as the point that was raised there, there is no protection for contractors. And frequently, especially with the subway project, where one of the contractors made some enormous disclosers, that contractor had no protections. Regarding the list of audit reports that are published on the csa web site, i think it would be useful to list all reports even though even the reports that are not shown on the site, so citizens can see what reports exist and if you know the report exists, you can sunshine say the Ethics Commission for that specific report. But i cant see any reason why an organization Whose Mission is transparency wouldnt post all reports even though they feel they cant post the actual report. Thank you. Any other Public Comment . Item six. Opportunity for Committee Members to comment or act on any matters within the committees jurisdiction. The first is fiscal year 2017 2018 work initiatives, of which there are seven. So well start with benchmarking and work through to the annual geo bond report. So i assume stop me if not, i will go through the list and brandon and or kristin can comment. Tonia referenced the benchmarking report and theyre finished and will have more presentation at the next meeting. Standardized templates, i gave you some of the differences at the last meeting that exist between the standardized templates and the things you liked the most in the year end report. And i havent done any further work on it since then, that has not progressed but i now have a staff person who i can ask to spend some time on it during the current quarter. Its complicated by the reporting issues that were referenced when mr. Carleton was asking and noted in the health bond. The priority is to get everybody paid and get reporting straightened out so they can do reporting. I havent wanted to push forward with julie dawson and other people in public works and bond managers who would be responsible because of the band width. Thats where it stands. Web site redesign, theres been work with our i. T staff people. There was a sheet in your packet that gives a sense of how a new design would work. Tiles is the basic answer where theres a lot of performance web sites which use a design like this and it looks to me like each bond program would be a clickble tile. And then we would add detail underneath that where you would be able to look at recent bond Program Reports. We could include some things like the graphs available from the work done in our fiscal year 16 17 report. And so at least those two things, which will be Better Organization of the bond Program Report that you already have and we know theres Quality Improvement work that needs to be done on sort of the administrative content of your site, where agendas and minutes are listed and that kind of thing. So maura, did you want to comment further . I dont think i included that in your packet. But ill get that to you today. Apologies. It was in the packet i had. Just a copy of an email from larry wilson which shows the tile design. Sorry. I thought i thought we were going to hire an agency for this. We are working on a contractor for this. So were not doing this, right . No. This web site and another section or two of the controllers web site that need design improvement. We have a contractor available to us and we need to proceed with a small contract to get it done. To execute it or decide what were going to put on there. Are we going to talk about the strategy. You give them a scope of work, they give you beta test designs to look at and move through it in that way. Okay. It seems these issues all sort of fit together, the benchmarking thing might inform what we want to put on there based on what other people put on there. The analysis on what people are going to the web site for right now, we want those easy to get to. I want to make sure theres actually an analysis discussion and not just a design. Yep. Anything else on that . Public Satisfaction Survey. So i have a little bit better progress to report on this one, included in your packet was we proceeded through a new rfp rpq we do to have a pool of contractors available to us, you and any City Department that wants to use it that do surveys, Stakeholder Engagement and outreach and choose a contractor in the rfq when we do the every other year big city survey. Thats the purpose of the pool. I provided a list of contractors who qualified under the pool to get a sense of who the market is, some of the names might look familiar to you. Some do a lot of survey work for the city, Library Open Hours site testing for the last two rounds. Fm3 as its called, they do a lot of survey work for the city. Just to give you a sense of the market there. Can i interrupt while were on this. Can you walk us through the four categories that you list in the top because it appears these are the categories were going to consider in evaluating qualified respondents. And how important it is to have a checkmark in all four or as many as possible. For the purpose of the stakeholder outreach work you want to do, i think anybody who qualifies in one of the first two columns could respond to your rsp. The difference there is subtle. Stakeholder outreach we make sure the people have experience doing survey work, survey design, sampling methodology. Language facility, submit work that shows their capacity in those areas. Stakeholder engagement, a lot of the same issues but also focus groups and facilitation services. And then, you know, so again when we issue the rfp, anyone who qualifies in column one or two could respond to it and no restriction to do so. Facilitation, this is us looking to have people we can hire to run Group Decision making processes which are sometimes requested by our clients or needed by us and the last column is city survey. This is the city survey you do once every three years. Every two years. Yep. Two other products that were sent to email and there are hard copies on the table over here. Those two products are a draft scope of work for what we might put out. Sorry, i should take a step back and refresh peoples memories, the desire on the part of the committee was to conduct a voter satisfaction or citizen or User Satisfaction survey of bond built facility to test peoples happiness in their User Experience and experience of the facility. This is a good thought of going out and asking people do they like the facility, is it clean and maintained, is the city getting what it pays for. Questions like that. So we needed to wait until we had pfrq survey providers available to do the work. We went back through in my staff and looked at the other surveys in the same area done by us and other city agencies. We have done surveys ourselves a couple of times. We did it on rec park facilities years ago. We have done it on street cleanliness. You go to the facility, you stop people and say would you be willing to spend five minutes filling out a survey for me. Sometimes people do it on site. Sometimes theyll do it online. Theres a couple of design issues that the survey contractors help us solve. So like i said,we have done it at rec parks and once for street cleanliness. We went through the rfps structured for those things. With marnis help here, my staff person in the back, pulled together the work to give you a sense of the kind of contract it would be. Its very simple. Its important not to overcomplicate this. I did try to put in for you some of the things which are definite. The survey design itself, its language reach which is standard and then some other options which you may or may not include. Just to call those things out, turn to the second page, i put down there deliverable five optional, develop a plan for existing standards and evaluations. You could do this or not do this. The most important thing is to be at the site asking people questions about how theyre experiencing the facility. But for rec park facility for example, we have stands in the city in our own Inspection Program that have to do with cleanliness, there may be other standards that im not aware of. Design, lighting, things like that where there is a city standard of some kind you would want to have in there and then ask your contractor to compare what theyre hearing from the public to the city standard. Maybe you just want answers to questions youre crafting and literature is an option. You can choose to do this or not, review related efforts, again, value or not. We could do this work opposed to a contractor. Those are some of the choices you can make in this. So hold that thought heres the draft scope of work for your consideration and then i passed out two sheets which youll need to do a little work to get to scope of work and a little bit of work to choose the facilities. I think what we have been talking about over a couple of meetings is completed projects in at least two park bonds, 2008 clean and safe and 2011 street repaving and safety bond. Marni did some work to put together the lists. Theyre not complete yet. I apologize for that. We need to check a couple of different sources to be sure that the project is ruled complete. We didnt have time to do the two tasks but wanted to let you know what universe you would choose if selecting those bonds. There are other options, we talked about choosing a larger facility like Public Safety building or Something Like that. I think we decided to focus on these types of areas. So thats where we stand. And im just thinking out loud on how to proceed with this. We can meet with liaisons and work through to completion on these two tasks and try to get the rfp issued before the next meeting so the work can proceed. Im very sure we can get it done before the end of the fiscal year, its just how much you want to leave to staff. I dont remember making a decision on facilities. No, you didnt. Okay, i really think we need to talk as a group about why were even doing this. What would we facilities in particular. I have no idea what we would do with the information. I think the suggestion, sounds familiar at least, i think ben had suggested and we had this discussion at a prior meeting is since this is the first time were doing it, this is the pilot, to choose some projects that have been recentl