The COVID Information Commons

Cliff Lynch: Welcome everybody. And we’ll be getting started in about Cliff Lynch: 90 seconds to two minutes. Cliff Lynch: Welcome, everybody. Thank you for joining us and we should be getting started in about Cliff Lynch: A minute or two and a half. Cliff Lynch: Thank you for joining us and we’ll get started in about half a minute. Cliff Lynch: Alright, um, I think it’s about time to get started. So let me. You’re welcome everybody to this last session of the second day of the CNI Cliff Lynch: virtual meeting.

I’m Cliff Lynch, the director of CNI and I just have a few quick introductory things to cover before turning it over to our speakers. Cliff Lynch: We are recording this session, and this will be available after after the session is complete, through our usual channels, there is a chat box and you are welcome to use that. Feel free to introduce yourself, if you like, or to comment as the Cliff Lynch: As the presentations. Proceed. There is a question and answer tool on the bottom of your screen and Cliff Lynch: Please use that at any point during the presentation to queue up questions, Diane golden Burkhardt from CNI will be coming on after all the presentations and will moderate a question and answer session, based on the questions that come in there. Cliff Lynch: The only other thing I will mention is that normally these sessions do have closed captioning available but there seems to be a glitch with this one, we should have closed captioning on the recording and we may get it working here during the session if we don’t. My apologies.

04:58 - Cliff Lynch: I think that’s everything I need to say in terms of mechanics. At this point I just like to welcome our four speakers Nora Garza Sarah Bowman Jeremiah Trinidad Christiansen and Florence Henderson. And I’m particularly delighted to have Florence back Cliff Lynch: Many of you will know her work from when she was at the Internet to Cliff Lynch: Some of you will also know her earlier work from when she was at IBM, but she is now the executive director of the Northeast big data innovation hub based at Columbia. Cliff Lynch: And one of the things that I hope will cover at least a little bit today is the role of the big data innovation hubs, which I don’t think is this widely known among the CNI community as it probably should be Cliff Lynch: This is an important strategic NSF initiative that actually touches on lots and lots of different projects. Cliff Lynch: The primary focus, though, of this presentation is going to be the very rapid creation of the covert information commons, which has been a sort of an amazing collaborative effort.

06:25 - Cliff Lynch: Like so many others that was born out of necessity is we’ve tried to respond to the challenges of the pandemic. And with that, I believe, Florence is going to start off the presentation. So I will just thank our speakers, one more time and hand it over to Florence. Florence Hudson: Thank you very much, Cliff. And it’s wonderful to be back with this CNI team. I’ve participated before, as you mentioned, and it’s always a pleasure. So thank you very much for having us today.

06:58 - Florence Hudson: Today we’re going to be talking about the code information comments and I’m joined by two of the code research principal investigators that have been awarded by NSF with code rapid grants and Jeremiah who leads our libraries team. Florence Hudson: At the Columbia University. So the coven information commons, it serves as an open resource to explore NSF funded research addressing the code 19 pandemic. Florence Hudson: And that was particularly what we were asked to do we received this code rapid grant Florence Hudson: It is is funded by the NSF convergence accelerator program, which is one of the newer programs which provides for multi disciplinary and interdisciplinary research across multiple of the domains that NSF funds. Next chart please. Florence Hudson: So our agenda today, I’ll give you a quick overview of the kick is, we call it the covert information comments.

07:48 - Florence Hudson: Then Jeremiah will take us through the library’s role and we thought this was a great example of partnership between researchers information technology, the big data hubs and the libraries. Florence Hudson: Than two of our principal investigators that we’ve spoken on our Florence Hudson: Community lightning talks are going to give their perspective and actually do their lightning talks so you can see the range of research that’s available through the coven info Commons and the community. Florence Hudson: Then I’m going to take you through a quick run through of the search and discovery mechanisms we have in a coven info calm and and then we’ll do our Q AMP. A which will all participate in next short, please. Florence Hudson: So, as Chris mentioned, the Northeast big data hub, which I’m the executive director for is one of four big data hubs across the US that are funded by the National Science Foundation, beginning in 2015 Florence Hudson: And then we got our next towards in 2019 so I leave the Northeast big data innovation hub at Columbia University representing the Northeast us Florence Hudson: The South big data hub, the West, big data hub in the Midwest, big data hub.

Similarly, are headquartered in a couple of universities and then 08:52 - Florence Hudson: Serve their entire region and this coven information commons was a collaborative project across the four big data hubs. Florence Hudson: And that’s part of what we’re trying to do more and more as hold hands and work together to support our community around the challenges and opportunities in big data. Next chart please. Florence Hudson: So a little bit of an overview. So the coven information commons was created to increase accessibility to information, something we all care about in CNI Florence Hudson: And this is specifically regarding the NSF code rapid Research Awards, which are rapid response research grants. So when there’s an emergency. Florence Hudson: They create these rapid awards, it could be for floods, fires pandemics, things like that.

And so that’s what it was funded for initially 09:34 - Florence Hudson: And now includes information for additional NSF grants that have coded as a program name or program reference code for those of you who know the innards of NSF parlance. Florence Hudson: And it facilitates knowledge sharing and collaboration across these coded research efforts, it really started as, you know, just give access to all this rapid information in one place and it’s evolved so much as a portal. Florence Hudson: As an international collaboration mechanism which will hear from Sarah. Florence Hudson: As well as a community of collaboration, which will hear more about from Nora, and from Sarah. So it’s a resource for anybody around the planet. As you can tell by the international nature. Florence Hudson: And we have researched her students decision makers from academia government not for profits everywhere really working together to leverage these research findings.

10:20 - Florence Hudson: And to accelerate research to mitigate the societal impacts of of the pandemic we organized relevant information in multiple ways. If you could just go back one chart and Florence Hudson: It allows us to look by research topic institutions geography and we recently on October 30 launched an upgrade that links to PCI and research team data. Next chart please. Florence Hudson: So this is a quick timeline, we, we were getting the award. We were awarded Florence Hudson: In May of 2020 and we launched to search mechanisms, a simple search mechanism into the NSF database as well as this cool machine learning generated map capability. Florence Hudson: Then in phase two, which we launched between July and and October, because we had the first launch in July, we delivered more search mechanisms we have Florence Hudson: We have a Slack channel. We have a number of other deliverables.

We have over 40 data sets that are actually shown and linked to 11:17 - Florence Hudson: Four around the world as well as these monthly kogut info commons webinars kick webinars and we actually did this PII survey to get more data that we’ve included Florence Hudson: In in the database, which is very valuable. And now we’re looking at next steps. And we’re going to talk more about that. Next chart please. Florence Hudson: So coven info commons is a portal and it’s a community. Florence Hudson: So in the community, what we’ve done is when we first were going to do the launch webinar in July, we said, you know, maybe we can ask a couple of P eyes to present and I actually begged one of them because I didn’t know if anyone would say yes. Florence Hudson: Then before we knew it 40 more people offer. So he said, oh my goodness, I guess they really want to do this. So in the launch webinar in July.

11:59 - Florence Hudson: We said, well, do your enthusiasm, you are now part of the covert info commons community and will have monthly webinars until we get through all these lightning talks. Florence Hudson: An hour scheduled at least through March. So we have hundreds of people in the in the community and you can sign up for it by going to if covert info commons.net which is our website. Florence Hudson: 73 members in our Slack channel and these monthly calls Florence Hudson: And Nora graciously, and our sep tember call when she presented, she said this, that we bottled it and we have in our presentations. Now I’m sure she’ll talk more about it, but Florence Hudson: She said your site and the ability to come together as marvelous.

I thank you, especially for thinking about this and bringing us together and people will be able to use your 12:36 - Florence Hudson: Site as a proper safe to information source. And that’s really what we were hoping for. Florence Hudson: So, as I say, as an Italian I nearly cried when she said that, but we’re so delighted that was the feeling. Next chart please. And this is a totally open community. Florence Hudson: This is just quickly to see that. And you can have these charts. I put them into the Dropbox. Florence Hudson: That these are an example of some of the researchers that presented Nora was in the first one. You can see the range here from UT Arlington to Princeton to UCSD.

13:03 - Florence Hudson: University of Southern California, all different directorates. Next chart please. And then in October. Sarah’s is presenting today presented Florence Hudson: And it was interesting because during her talk there was someone I said I asked a question, and I go, Sarah. Peter asked a question, because I have like three others I’m answering right now. There was so much real Florence Hudson: real time collaboration. It was so exciting. Next chart please. And then our next webinar is this Friday tomorrow, November 13 and you’re welcome to join that as well. Just go to covidien Phil collins.net and register under events. Next chart please.

13:33 - Florence Hudson: So since we launched it as a portal to find rapid grants. Florence Hudson: We also created a couple of other things. One is we created a tab called meet the researchers really trying to humanize this so we have human to human collaboration going on. Florence Hudson: And so what we’ve done is we’ve taken the separate lightning talks from each of the p eyes that have done them. And we’ve put them in separately. We also connect them to their PII profile which we’ll talk about next chart please.

13:58 - Florence Hudson: We’ve also started doing interviews with some of the researchers to ask them five questions about why are you doing this research and what have you found what are you hoping to find what collaboration opportunities do you have Florence Hudson: Once again humanizing it and creating the ability for people to find each other to enable each other’s research and collaborate together. Next chart please. Florence Hudson: So these are a few other resources on the kick website and you can go at anytime you can be playing with it and another screen right now if you want covert info commons.net Florence Hudson: So we have under our resources we have webinars videos, including a user tutorial for the lingo 4G machine learning explore an app that I’ll go through a little bit more Florence Hudson: Than we have the July September, October, and we’ll add the videos as they come about. Next chart please. Florence Hudson: We also have research funding opportunities around the globe US and international. Next chart please. Florence Hudson: We also have over 40 data sets from six continents.

How cool is that people send them to us because they know we have this portal. Next chart please. Florence Hudson: We also have groups and guides organizations and networks guides and references next shirt, please. Florence Hudson: And so all of that is provided by people who say hey you know include us and we’re delighted. So if you know of code related resources, feel free to go to code info commons.net and we actually have Florence Hudson: A form that you can fill in, to provide data under the events we have prior events.

You can see past events that we have upcoming events. We have these monthly webinars, because we have so many cool lightning talks lined up next, short, please. Florence Hudson: And this is our project team so Jeanette wing is RPI at Columbia University. She leads the Data Science Institute. I work in her team. Florence Hudson: And then my other three colleagues that lead the other big data hubs Meredith Lee for the West, big data hub john McMullan for the Midwest and Renata Rawlings Goss for the sale pub. Florence Hudson: Katie anonymous and operations manager Helen Yang is one of our wonderful students.

She’s a junior at Columbia and she’s done a lot of the updates to the database. Florence Hudson: And then our Columbia University Libraries TEAM AND IT teams and all together, we were able to pull this off, which is kind of incredible and we’re really delighted about it and now we’re looking at our next steps. Next shirt, please. Florence Hudson: So now I’d like to hand it over to Jeremiah Florence Hudson: So he can talk about the libraries will, and they were really important for us because they help us see some of the information science aspects that aren’t as obvious to some of the rest of us. So, Jeremiah, Jeremiah Trinidad-Christensen: Okay, thank you. Lawrence. So, this is this was a pretty good project to be working on it was at the time.

16:28 - Jeremiah Trinidad-Christensen: Something working with something that did not fully exist yet. So a lot of the words were in the in the same process as we were Jeremiah Trinidad-Christensen: So, it meant a lot of room to kind of think and explore and play to understand what are the possibilities. Jeremiah Trinidad-Christensen: Of what this could be other libraries team that we, that we have listed here. All of us are within the digital scholarship unit within Columbia University Libraries. All of us have some level of expertise with institutional repositories.

16:58 - Jeremiah Trinidad-Christensen: Working with data metadata data management even building web applications as the role that we were asked to do is was to to be able to come in. Explore and think of ways for discovery for the information that that that was there, we did know. Jeremiah Trinidad-Christensen: From the beginning, that that we should be thinking very flexible, that this is something that Jeremiah Trinidad-Christensen: Although it was it was geared toward the the rapid awards, there were, there was room for growth in here. So in all of our thinking we needed to keep that in mind. Jeremiah Trinidad-Christensen: That it wasn’t just for this information alone.

So as part of that we needed to understand what what data was available from NSF 17:42 - Jeremiah Trinidad-Christensen: We need to also think through what are the additional information that might enhance in the, the, the data that’s available through and stuff that might be helpful. Jeremiah Trinidad-Christensen: So we took a look at the existing data, the data came through both API and XML. We took a close look at what was available there the XML actually had a lot more information there. But it wasn’t updated as as as quickly Jeremiah Trinidad-Christensen: Whereas the the information from the API was Jeremiah Trinidad-Christensen: Very regularly updated so we compared we assess for the completeness. We try to understand any errors or any problems that might might be there that might exist we mapped out the various fields or different relationships thought through how each one might Jeremiah Trinidad-Christensen: help somebody explore if they if they saw it readily in the in the front in the back or what all those relationships.

We also took a look at additional information that we might be able to pull in 18:47 - Jeremiah Trinidad-Christensen: To enhance us. We thought through, you know, if we brought in connected to to information out of orchid. What would that look like connect to two other other research that that each individual Jeremiah Trinidad-Christensen: Researcher was had already done, or will do in the future, making connections to wiki data, for instance, for the institutions or other information. Jeremiah Trinidad-Christensen: That might be pulled out of their out of like the abstracts and be able to have that final insurable. Same thing with geo names, whether or not we wanted to explore looking at building an Jeremiah Trinidad-Christensen: interactive map that that lets people make selections custom selections based off of whatever area they wanted to draw on the map.

19:30 - Jeremiah Trinidad-Christensen: All of these kind of things we are looking at. Same thing with controlled vocabularies to Jeremiah Trinidad-Christensen: And then from the P is we knew that the survey was going to go out. So we need to think through what are the, what’s the information we wanted to get that might be really helpful and the orchid ideas were one of those, make sure that we connected the individuals that were listed there. Jeremiah Trinidad-Christensen: With who they are. Jeremiah Trinidad-Christensen: Same thing with we’re trying to understand any unstructured keywords that might happen.

20:00 - Jeremiah Trinidad-Christensen: How do the researchers see their own work, how do they describe their own work. What are the key words they applied to the to that to that research and also the output to since this was so early on. Jeremiah Trinidad-Christensen: Most of the there were no outputs. Jeremiah Trinidad-Christensen: At the time when we’re looking at this, so really had to get an understanding of what we were looking at and how to display that through the site. Jeremiah Trinidad-Christensen: So after we got roughly about 100 or so responses from the survey the library team took took a dive into the data and just focus Jeremiah Trinidad-Christensen: Very specifically on the any of the websites that were that were named in there as well as what type of output was going to be available.

We did a kind of cursory pass of this to try and kind of organize ourselves. So this is the this these couple charts are our Jeremiah Trinidad-Christensen: show you kind of what we were thinking at the moment. So with the the URLs. Jeremiah Trinidad-Christensen: We got URLs in there that some of these were associated with university somewhere.org dot coms somewhere. We’re in GitHub, we pull up the, the Google ones just because Jeremiah Trinidad-Christensen: Since these are these are things to exist. They did not exist at the moment. Jeremiah Trinidad-Christensen: Some of the Google sites where we’re might be associated with with log information from for from a particular institution or just from Jeremiah Trinidad-Christensen: A few individuals. So we knew we might have to look at these later.

But we need to understand how many of these we we were that we had 21:36 - Jeremiah Trinidad-Christensen: At that moment, we also wanted to, to look at the output itself, and there was such an amazing variety of output everything from data sets to two Jeremiah Trinidad-Christensen: Things that would exist in social media that that might be a blog that might be a video that might be some kind of a conference or some kind of Jeremiah Trinidad-Christensen: Well, just about anything you could think of it was it was quite expansive and really, really interesting to see. Jeremiah Trinidad-Christensen: But with all of that we needed to keep in mind that Jeremiah Trinidad-Christensen: Once again, this had to be something that is simple, that it’s clean fast inflexible something that whatever is built off of this, it could be the building blocks for for for the future for whatever it’s going to become also something very lightweight that Jeremiah Trinidad-Christensen: That if we if we build something very complex in the back end, we may be stuck with that particular back end so Jeremiah Trinidad-Christensen: We put the pieces in place for being able to expand on this one, and this is this is what you see right here. So everything from the the keywords on here that that link out and make connections, but that’s it for the library side of it. So I’m gonna pass it on. Florence Hudson: Thank you very much, sharing my. We really appreciate it and your partnership has been great.

22:56 - Florence Hudson: So now I’d like to pass it over to our RP is the code rapid P eyes that are joining us today. They’re going to do their lightning talks and give us a little perspective on this coded info comments. Florence Hudson: First first will be there. Dr. Sarah EJ Bowman associate research professor in the Department of Biochemistry at the University of Buffalo. And then Dr. Nora Garza from Laredo College in Texas, Sarah, take it away. Jeremiah and extract, please. Sarah Bowman: Okay, great. So can you all hear me. Yes. Sarah Bowman: Great.

So I’m going to tell you a little bit about some of the work that we’re doing in the lab that I direct 23:32 - Sarah Bowman: I actually run a crystallization center for structural biology at Hartman Woodward Medical Research Institute, which is a small nonprofit research institute and I’m also affiliated with University of Buffalo. Go ahead and. Next slide. Sarah Bowman: So I’m a structural biologists, we do structural biology and what that Sarah Bowman: What that means and what that has to do with the code 19 pandemic. Is that what we’re trying to kind of figure out is what all of the different pieces and parts of the source code to virus really look like Sarah Bowman: So many of you are probably familiar with this type of figure which looks large and scary and it’s got these big growth spikes on it. Well, that spike is actually a protein. Sarah Bowman: And it’s on the surface of the source code to Varian, and it’s the protein that interacts with human proteins. Sarah Bowman: And so what we do in social biology is try to understand what these things actually look like because it helps us to figure out how to design vaccines and drugs and so on and so forth to go on to the next slide. Sarah Bowman: Next slide. Sarah Bowman: Can you guys still hear me. Sarah Bowman: Okay, great. Let’s go back one. Sorry.

So structural biology is actually somewhat difficult to do. And in part, this is because these things that we study are incredibly small. And so I like to give people a framework. Sarah Bowman: For how how big were actually looking at. So if you, if you think about the width of a single human hair, which is obviously out not not quite to scale here. Sarah Bowman: But with then we ramp down to a single grain of pollen to one red blood cell to an aerosol droplet all the way down to that little pencil.

25:14 - Sarah Bowman: mark of a source code to variant. Now, what we’re trying to do in my lab is actually look at what do these things look like in the pieces and parts. And so if Sarah Bowman: If the small thing is is so small pieces and parts are even smaller. And so there’s a number of techniques that let us do this, go ahead and go to the next slide. Sarah Bowman: I’m sorry there. Sarah Bowman: So don’t worry we’re not going to actually talk about the source code to Gino but we do think about the source code to genome.

25:44 - Sarah Bowman: In terms of what are the different proteins that are encoded by it because those are the things that we’re trying to understand what they look like. Sarah Bowman: So the structural biology community and the whole scientific community as all of you are probably aware of have really Sarah Bowman: Worked together, you know, very quickly to try to understand a lot of these things. Sarah Bowman: And if you go to the next slide. The reason. Again, we’re trying to do that is because these are the proteins that actually our treatment targets. So if we can Sarah Bowman: Actually develop drugs that will bind to some of these proteins to stop them from acting, we can stop the virus.

If we can develop vaccines for the 26:21 - Sarah Bowman: That that bind the spike protein. And that’s actually part of what has been really great in the news lately is is is some of this work right. Sarah Bowman: So in structural biology. There’s a couple of techniques. Go ahead. And then the next slide, the one we work on in in the lab. I work with is Sarah Bowman: X ray crystallography. And the only thing you need to take out of this is that what we need to do x ray crystallography is to take our sample and turn it into a crystal.

And that turns out to be the bottleneck in the whole technique. And so, next slide. Sarah Bowman: So what we do in the crystallization center is we actually facilitate that happening. And the way we do that is in a high throughput way using Sarah Bowman: Well tray that is essentially the equivalent of what most labs doing the 96 well tray so it’s about 16 times that Sarah Bowman: And we have really extensive robotics imaging instruments and a lot of expertise, because we’ve been running for about 20 years. Next slide. Sarah Bowman: And so we were, we received our NSF rapid to essentially facilitate researchers from all over the country who were doing source code to research to try to understand the protein structures so they could send their samples to us. Sarah Bowman: We could screen them for crystallization conditions and then they could proceed with Sarah Bowman: With their studies.

So this is especially important in the source code to kind of situation because so many labs are running at really decrease capacity because of for safety. Right. So go ahead and. Next slide. Sarah Bowman: And so we’ve really become a major resource for we found a major resource for structural biology, but it’s been a really fascinating time to be doing this work with the source code to proteins. Sarah Bowman: So this is kind of some of the types of images that you get Sarah Bowman: Is to actually view the view that wells over time. And you can see crystals forming. Sometimes they look like little flowers like in this one. Sarah Bowman: We actually have multiple imaging techniques that help us to really determine those are crystals, one of the tricky things about this is so Sarah Bowman: That we have 1500 and 36 conditions.

We have a lot of different imaging over multiple types of images. And so we end up with about 14,000 images for each experiment. Sarah Bowman: And at the moment right now. People have to look through all of those. So one of the things our source or rapid funding funded. Next slide. Sarah Bowman: Was some great work by this fantastic post back student Ethan Sarah Bowman: Who essentially developed a graphical user interface to make it a lot easier to actually view these view these images.

28:54 - Sarah Bowman: And and be able to really interact and work with the data and he incorporated a really recent machine learning algorithm that identifies these crystal images. So we’ve submitted. Sarah Bowman: A paper describing the software. It’s on GitHub. And so available for people and he recently also got a poster prize for for this. So, next slide. Sarah Bowman: And we’ve also had just some tremendous success screening for source code to samples for users. So these are some of the examples. Sarah Bowman: In the top corner you see some crystals have one of the protein cases which is a main drug target in complex with a number of different inhibitors Sarah Bowman: The one with the little heart shaped picture that’s actually the main protease, the crystal structure that was solved by our collaborators at Oak Ridge National Laboratory and again these flower like crystals appear within you know a week.

29:47 - Sarah Bowman: And then we’ve we’ve actually just this week gotten some amazing results on one of the proteins that nobody else has actually had any structure structural work with. So we don’t have. I didn’t have any time I wasn’t able to put pictures up about that but Sarah Bowman: It’s really exciting. So next, next slide. Sarah Bowman: So it’s, it’s been really great to be part of the coven information comments as well because it really facilitates connections. And I think one of the really cool things about this is that Sarah Bowman: You know, there’s a lot of people who are doing a lot of different type of work. So we do structural biology. There’s people who are studying all kinds of different things.

30:25 - Sarah Bowman: Already I can say that that I have active stars collaborations. Sarah Bowman: So this is one of the things that I think Florence will be talking about a little bit, but essentially you can do searches with keywords to see who else is working on things that are kind of near you. And when I did a search earlier this week on Sarah Bowman: On this, I discovered. Oh, I’ve got, I actually have three active collaborations on source code two projects right in my little network already. And so it kind of gives you an indication of how well this this Sarah Bowman: Network is working. And then also, next slide.

30:59 - Sarah Bowman: I also just last week was contacted by somebody who found our work via the coven information comments website. Sarah Bowman: And it’s a graduate student in South Korea, who is doing research on how data open access during the coven 19 pandemic kind of compares to other types of things. Sarah Bowman: Her advisor is in the UK. And so we’re actually scheduling a time to talk about, about how all of this stuff works. And so I think that one of the Sarah Bowman: You know, exciting parts about what’s happening right now. Sarah Bowman: Is all of the connection that that is happening.

And that’s a, you know, that is being enabled by by kind of what’s happening with with the coven information comments so 31:40 - Sarah Bowman: Last slide is just my thank yous. So I’ll. Oh, I do want to mention there is a Slack channel for the coven information comments and we do have a special structural biology channel on that. Sarah Bowman: So if you’re interested in any of the things that we’re doing. You can come and chat, chat with me there or check out our website or our Twitter handle. So thank you. Florence Hudson: Sarah that was marvelous. Thank you so much for enunciating the international value. How cool is that you and I was just so cool and say that Sarah actually you created the structural biology channel within the Slack channel didn’t Sarah Bowman: Actually one of my Sarah Bowman: One of my co workers that hwy did Florence Hudson: That’s what Sarah Bowman: Who also has a rapid award. So Florence Hudson: Yeah. And so that’s a really good point that this is all community driven. So the community is finding this the international community is finding it. Florence Hudson: You can create your own like sub channels within the Slack channel to enable collaboration for your focus areas.

So we’re really trying to make this as community driven as as possible. Florence Hudson: And we’re so grateful that the community is getting benefit out of it. So thank you for sharing your story. Sarah being part of the community. Sarah Bowman: Thank you very much. Thanks for having Florence Hudson: Of course, hang on. We’re not done. And so, Nora, I’d like to pass it over to you. And as I mentioned, Nora was one of our first stars. Florence Hudson: In the code information commons community webinars and her her copia GABRIELLE IS ON THE LINE listening in as well. And I’m sure she’ll want to thank her. She’s been wonderful to work with. So, Nora, take it away.

33:07 - Nora Garza: Thank you so much, and Nora Garza: At our at our most recent meeting with our students and our faculty Nora Garza: Last week, they said, well, thank you to Dr. Guys, and I said, No, thank you to the National Science Foundation for funding us Nora Garza: And a big thank you to the information commons, which has made it even more exciting for us and for our students. It’s nice to do multi generational and so my star is Nora Garza: Who is a former student here at the radio college and now she is our undergraduate research coordinator. So our project is using the coven data that’s already there in the city of Laredo Nora Garza: Website to teach quantitative reasoning skills to undergraduate Hispanic students Laredo, Texas is right on the border with Mexico. The US came to us so Nora Garza: We’ve been here a long time, and our college is about 96% Hispanic students can you change the slide please.

34:07 - Nora Garza: So we are working with six faculty members and of course if you cannot say it enough. Nora Garza: There’s a huge stress on all undergraduate students, particularly those pursuing a STEM career. THESE ARE HANDS ON KIND OF STUDENTS. They’re very high energy Nora Garza: And with a coven you know that we’re interruptions in their undergraduate research opportunities they would do bird counts. They would work with simulators. That would work with our faculty and all of that is gone for now. Nora Garza: We’re very safe, but we are not meeting face to face.

So it’s important to retain these students in their classes, even if we have to do it virtually so we’re going to be using that. Can you change the slide please. Nora Garza: When we first got the opportunity to actually write the grant. And I’ve mentioned it to Florence I had read Harare, and was reading Heath, the Heath Brothers. Nora Garza: And somewhere there they mentioned something about when the impossible happens. Think of the possible Nora Garza: And so I like to be on the positive side of everything I wanted to set up and implement a data analysis research experience to improve quantitative reasoning.

35:21 - Nora Garza: Thanks to NSF and our meeting for the HSI a project principal investigators, we met, I met Dr. Esther Wilder from New York. Nora Garza: And she’s been amazing. And she and I in conversation said, you know, when students can really use the terminology practice it. Nora Garza: You know, everyone has impressed one they get to practice our vocabulary and the end understanding and using quantitative reasoning is really important. Nora Garza: So we wanted to create a professional development opportunity for our faculty Nora Garza: And any quantitative reasoning and then we wanted to evaluate the projects impact on our students quantitative reasoning. So we have faculty from chemistry for math. Nora Garza: From psychology and from nursing and they’re working with it 14 students and all their projects are related to the coven so if you change the slide please.

36:20 - Nora Garza: This is what they used to do before. Obviously, they’re not out on the river together. Nora Garza: But this is dr Mang and his students and and soon to be Dr. Linda Martinez, who I think it’s at University of Buffalo. Nora Garza: Faculty Development training we are reading a book called math for life. And we’re using it with our students or one of our other teachers is a teacher statistics. So this is great. Change the slide please.

36:46 - Nora Garza: So you can see there the Laredo city of Laredo Nora Garza: Dashboard and Nora Garza: Excuse me, we are, these are the posters that are used to students used to prepare before they’re going to be preparing them again for the projects that they’re doing. Change the slide please. Nora Garza: So you have there that we’re using the book we have our faculty. We had this is a second or third meeting that we have added with the students. Nora Garza: And so this past week to our math instructors Dr got an answer and Mr guidelines up brothers met both math teachers. Nora Garza: They talked to them about the why intercept and what we were trying to get was what was the rate of change.

We wanted for coven 37:35 - Nora Garza: And that would be to be zero to flatten the curve. So they’re talking about these things graphing and charting and it’s it’s exciting for our students. Nora Garza: We are keeping them motivated and that was my big, big worry. I don’t know about other universities, but a college a community college with STEM students and Nora Garza: The situation that we’re all living you know we have had many students drop out because they have to do other things to take care of their families. Next slide. Nora Garza: But I law. Nora Garza: coven information comments community welcomed us we became a part of them and our students and our faculty are using everything that’s there.

38:20 - Nora Garza: And even to write future grants, we are connecting they’re connecting the dots. It is easy to access. If you want to find out any other person that’s working on the rapid Nora Garza: grads four minutes. If you can find it there. The research, research runs the gamut from research on the actual virus, like the one that you just saw. Nora Garza: To the educator and the students reactions to cove it you know what has been happening. So there’s all kinds of things that you can access their Nora Garza: A quick perusal will put you in touch with information.

38:54 - Nora Garza: On informal Science Learning during these trying times. And of course, Gabrielle last Elise, who is working. Nora Garza: Hand in hand with our students 10 hours a week, each student works 10 at least 10 hours a week on their research and being able to connect to the information commons is very exciting. I know they’re all looking forward to tomorrow’s lightning rod lightning talks next Nora Garza: And of course, if you have any information on any would like any more information you can contact me at Mr Garza at a rate of.edu which you can find more information, of course, on the on the information comments. Thank you very much.

39:35 - Florence Hudson: Thank you, Nora AND THANK YOU, GABBY who’s been in the background and foreground of this whole thing too, so Florence Hudson: That was wonderful. Both of you are just so inspiring and this is an example of what the coven info comments is bringing together. We’re very fortunate. Next chart please. Florence Hudson: So now I’m going to give you a quick run through of some of the search and discovery mechanisms that our colleagues have been talking about. Next chart. Florence Hudson: So, as we mentioned, when we first launched this in July, or a minimal viable product after we got the award in May. Florence Hudson: We actually launched it with to search mechanisms.

The two that you see in the pink boxes on the bottom. Florence Hudson: So the code research explore machine learning generated maps as well as a customized search by NSF directorate and what we try to do always being community driven as we listen to what people want. Florence Hudson: So as you could see, Sarah was using machine learning generated maps. I’ll show you more about that so you know where that came from. Florence Hudson: And then I’ve had other P eyes, who have said that they actually like going and hanging out, like in one of the director.

It’s just to see what type of 40:36 - Florence Hudson: You see an orange, not in your head to see what type of research is being funded and as Nora mentioned, you could look for gaps you could look for continuity, you could create a constellation of stars. Florence Hudson: You know, how could this all work together. And so what we try to do is, you know, provide different tools that are valuable for different people. Florence Hudson: And one of the things we’re planning on is a student Hackathon or challenge that we hope to announce in December to do some of the things Florence Hudson: That you said your students already doing is have them leverage these different search tools to find opportunities for collaboration. Florence Hudson: To bring things together, things that are not being done yet. You know what’s missing.

41:12 - Florence Hudson: And to give us some feedback on the different search mechanisms to see if we should consolidate it or just keep all three of them there because we’ve been, you know, building this over time. Florence Hudson: And we just announced the the NSF code awards and PII database, which is really fun. I’ll take you through all of these. Next chart please. Florence Hudson: So the one that we just launched as as Florence Hudson: Jeremiah was talking about too is the awards MPI database. And as you can see, when you go into it, you can get a list of all the awards. You can do it alphabetically by PGi by institution.

41:44 - Florence Hudson: And then all these things are the right are all the different filters, you can use to actually search the awards. Florence Hudson: You could look by NSF Director it by division by the institution themselves likely radio College, University of Buffalo, the state that they’re in our territory. We have Puerto Rico in there as well. Florence Hudson: By region that for big data hub regions P I name the program officer at NSF, the program name reference codes, all sorts of stuff so that you can actually go in and zero and and what you’re looking for. Next chart please. Florence Hudson: So when you go into this, you can click on one of those awards and can see all this information about the award. We were talking about Florence Hudson: And the really exciting part is you can also click on. You can see how the name of the p is highlighted. So we’ll click on Peter rose. Next chart please.

42:29 - Florence Hudson: And you can see this is his PA profile. And so, Nora has won Sarah has one, and he sent us a fair amount of information we did the first PII survey. Florence Hudson: You can see his institution is email his orchid ID, which is a hot link to his orchid page so you can see what else he’s had going on. Florence Hudson: The websites. He provides that either have results of his research or related to his research once they do a Florence Hudson: lightning talk we add that here as well. So those are also in Sarah’s and Norris. Florence Hudson: And then, we not only have the list of rapid awards from them, have more than one rapid covert award, but we have, what do they expect their Florence Hudson: Scientific outputs to be the outputs from their research collaboration opportunities and the keywords, they use to describe what they do.

43:10 - Florence Hudson: Because it could be that their keywords, you know, based on the NSF database, but I always like to ask people, How do you classify it, you know, what is it that you’re doing because it may be a different way of looking at things. Next chart please. Florence Hudson: So that’s the P i database. And you could go in and use it as much as you like the next one. This is a very simple thing, but Florence Hudson: Actually the machine learning maps. I’ll do first, and I’ll do the simple one at the bottom. So these are the two first mechanisms. Florence Hudson: That machine learning maps for the Reese Kirby research explorer and then the NSF directorate version. So, next chart please. Florence Hudson: So the machine learning generated maps are based on a tool called lingo 4G explorer and when you go into it, you can actually see Florence Hudson: A topographical map of the NSF rapid awards and the topical areas. So this is interesting. And if you look at it and you can see that as an example protein at the bottom. Florence Hudson: That looks like a big mountain. So there’s a lot going on down there or social distancing or surveys or academic learning Florence Hudson: And so you could go in and you can zoom in on this and you can also click on any of these little dots.

44:12 - Florence Hudson: And actually learn more about that research and whenever you click on any of the awards, as well as in the next page, you get this information on the right you can choose what fields you want the name of the award. Florence Hudson: The institution, the API name, the amount of money they received their email address to state they’re in, you know, the program officer. There’s all sorts of information you can pull up based on what filter you want to use. Next chart please. Florence Hudson: Then there’s also this view, and this is like the one that Sarah showed where these are the tree maps versus the topographical map. Next chart please. Florence Hudson: And as you go into this, you can actually customize the view you have so you can even see here, we can say I want to color this by this cute little cog up here.

44:52 - Florence Hudson: You can call her by state. I’d like to size it by the amount of see how much money they got a lot of the rapids about 200 K. So it’s pretty consistent. Florence Hudson: labeled by institution. I want to highlight the same color. And let’s see what that does for us. Next chart please. Florence Hudson: So here I click on one of the institutions in California. So since I said I want to see it by state. I’m going to see color. Florence Hudson: State by color and highlight everything the same color. When I click on a California institution, all the California rapid PII rapid grants come up.

How interesting is that, so if you want to see who else is in your system or in your state. This is the way to do that. Next chart please. Florence Hudson: Then you can also look at awards, you know, by institution and in California. Florence Hudson: So this would be I asked for the view by institution and over here in the query, I can say, and state, California. So it shows me just California, the last page, they call it in the California ones, but there was a whole bunch of them here, I see just the ones in California. Next slide please. Florence Hudson: And so now I can customize the view by P eyes so I look on my label API. Next chart please.

45:59 - Florence Hudson: And then I can actually go in and go for a particular area. So I could say end machine learning or end machine and learning the joy, a Boolean in this case is you get to try it different ways and see what you get. Florence Hudson: So, you know, the joy of programming for those has been in it. And so these are all of the ones that are related to machine learning. Next chart please. Florence Hudson: And then I could look at, you know, the P eyes that are doing the machine learning. And then, next chart please.

46:28 - Florence Hudson: I can go into the other search mechanism, which would be the NSF Directorate. So here you can click on these icons to find the code rapid grants by NSF Director Florence Hudson: The director, it’s biological sciences Computer and Information Science, Engineering, called sighs often Florence Hudson: Education Human Resources engineering geosciences mathematical and physical sciences social behavioral economics sciences and office of the director Florence Hudson: And they all represent code rapid grants. So if I click on Office of the Director. Next chart. You can see this is actually our code info com Florence Hudson: So it shows you a list of all those and the director it and you can click on this one. Next chart please. Florence Hudson: And it gives you a similar information as to what we have in the API and awards database, we’ve integrated this with the PII information, including the abstract and a lot of it useful information. Next chart please. Florence Hudson: So that’s a quick overview of the coven information commons.

The portal, the community, the API’s that are working in it the libraries team and how they enabled us to put the information together. Florence Hudson: The best way that we could to gather the right information from the API’s to make it more useful. Florence Hudson: And one of the things that we do is really try to follow the fair principles, you know, for data in the coven information comments, making it find double accessible interoperable usable. And actually, we’re going to be presenting at the code data go fair conference virtually in Paris only Florence Hudson: Virtually November 30 and we have nine of our P is presenting there as well. Florence Hudson: So find double is what F stands for.

So you can find all the NSF rapid awards and we’ve actually added eager as an SPI our business TT Rs. For those of you who know some of that nomenclature and we’re looking to add more over the next year. Florence Hudson: That are coven related and you can find them, you know, easily and context, it’s very accessible people using it around the planet and you can get all these different elements that we talked about. Florence Hudson: interoperable from a researcher perspective, we’re actually thinking about the next stage of the code that info commons and Jeremiah and I were talking about it earlier today. Florence Hudson: What we’re thinking of how cool would it be, we haven’t figured out how to do this yet.

If we that we could actually crawl the data and help Sarah. Florence Hudson: Use her data to find other data in the same type of area or for Nora and her students to be able to have their data find data like what, how is it happening in other states and other HS eyes. Florence Hudson: You know, how does this compare how cool would that be so we don’t know who can help us code it, but we have it. We have $1 and a dream. Florence Hudson: And so if anyone knows, anyone who can help us with that Jeremiah are saying we’re open, you know, to that. And the data is all reusable, you can download it from the explorer tool from the API award database.

49:03 - Florence Hudson: You know, there’s a CSV file with all the information and as well. You can download in, you know, Excel or XML all sorts of formats from the director level information and we’re looking to make that even more useful into the future. Next chart please. Florence Hudson: So the next steps. Use it, you know, it’s your tax dollars at work. You may as well use it and your kids can use it. Your grandmother can use it. Your friends and Italy, anyone can use it. Florence Hudson: Showing the coven info calm and Slack channel if you’d like. It’s on the cake website. If you go to the covert info common stat net. There’s a thing at the bottom that says join the Slack channel. Florence Hudson: You can sign up for the community and the kick kick events. There’s one tomorrow. There’s the Bitly for it.

49:42 - Florence Hudson: And, you know, help us have researchers collaborate with each other and with students. Florence Hudson: We’re going to be doing the international fair convergence symposium. As I mentioned at the end of November. Florence Hudson: 1300 to 1500 UTC which is now 10 to 12 Eastern. It was 11 to one when they sent me the invitation. So luckily we figured that out before November 30 Florence Hudson: Because time clocks changed but UTC doesn’t. And you can email us your input or questions at info at coven info comments.net. So, next chart so on.

Now we have time for Q AMP a 50:12 - Florence Hudson: And we’d be happy to take any of your questions, or your answers we accept both as I usually say, so if there any comments. I don’t know if there’s anything in the chat Clifford Diane Diane Goldenberg-Hart (CNI): Right. Thanks. Florence, um, Diane Goldenberg-Hart (CNI): Thank you so well first of all, thank you for that tremendous presentation, all of you. This is very exciting tool and just Diane Goldenberg-Hart (CNI): Really remarkable the kinds of synergies that can occur when you bring different sectors of the community together. Diane Goldenberg-Hart (CNI): With their various needs and interests, it’s, it’s the hub is really exciting.

So thank you for this wonderful demonstration of the power of that and my apologies to everyone, I’m having some technical difficulties so they’ll just get my pretty flower for now and not my pretty face. Diane Goldenberg-Hart (CNI): So with that would like to invite our attendees to share with us. Any comments or questions you might have. I think Diane Goldenberg-Hart (CNI): Florence and Jeremiah throughout the challenge there. So if we have anyone in attendance. Who would like to take them up on that offer, please let us know. Diane Goldenberg-Hart (CNI): And I’ll just ask while we’re waiting for attendees to weigh in.

51:27 - Diane Goldenberg-Hart (CNI): I’m just wondering what kind of outreach, are you doing to Diane Goldenberg-Hart (CNI): You know, who, who, who are you hoping to draw in how are you reaching them other than coming to speak at sea and I, of course, what, how have you been reaching people Florence Hudson: So I we try to do it early and often. So we presented at the academic Data Science Alliance leadership summit. Florence Hudson: Which was about a month ago, I think. And so that was a really good discussion I presented this recently at the the UC Santa Barbara responsible machine learning summit. So people are reaching out, which is wonderful. Florence Hudson: You know, as I mentioned, will be speaking at the international NGO fair conference later this month.

And I think that’s a great opportunity to increase collaboration across the world. Florence Hudson: I’m hoping that added that we have more researchers that start using it on purpose, like the ones that you know communicating with Sarah. Florence Hudson: I hope that we find more data sets, maybe award databases you know that we could look at and try to link to Florence Hudson: You know, this isn’t even though we’re called The Big Data hubs where data innovation hubs. We don’t have like big data, you know, storage networks and things like that where we link to everything. Florence Hudson: So we’re more like a collaboration hub. But those are some of the things that we’re doing.

We’re going to be reaching out to all the 990 P is 52:48 - Florence Hudson: That are in the, the new awards database with the PII information to get more of them to give us their survey data so we can enrich it more Florence Hudson: We’re very, we’re very lucky that some of the p eyes actually did fill out the survey before they could see what we were going to do with the data. But now that they can see I’m hoping that some of them say, Oh yeah, this is a great idea. Florence Hudson: We had 170 that you know that listened and just gave it to us, which we really grateful for. And when we asked for it. Florence Hudson: So we’re reaching out as much as we can. I’ve spoken on the international coven Florence Hudson: research seminar that occurs monthly that’s led by the Pittsburgh supercomputer center with collaborators from around the world.

53:24 - Florence Hudson: And we’ve also just started to talking with a coven 19 HBC consortium, which works with exceed in price in the US and in the EU. Florence Hudson: So we’re going wide, you know, we want as many people to use this as possible. I, I like to say this is your tax dollars at work, so use it. Florence Hudson: And help us to help the research move forward because there’s a lot to figure out, you know, when I started this project. Florence Hudson: NSF actually came to us in March and said, hey, we’re thinking of this thing.

And I was like, wow, what’s that about, you know, we started talking about it and I actually at the time did a quick 53:57 - Florence Hudson: A quick lit review, so to speak, and I found that in 1985 NSF funded, I think it was the second biennial coronavirus workshop in California. Florence Hudson: I know, right. And I’m like, where’s that data. So there’s actually a spring republication with some of the information in 1986 Florence Hudson: So as a researcher, I look at that and go, man. What’s the same what’s different. Florence Hudson: What did somebody know and i actually reached out. I tried to find one of the p eyes, but they were probably my age, back then, but one of them was related to Florence Hudson: One of the California universities and they were in touch with him as a as a retiree and I Florence Hudson: Had to send them a note to see if I could interview him or something and I haven’t heard back yet. But now that we have come so far with this. I’m going to see if maybe Florence Hudson: He will talk to us.

How cool would that be, you know, that we can share but some of the things are consistent, you know, you read it says, wash your hands, you know, 54:58 - Florence Hudson: Social distance. This was in 1986. So this is not rocket science, as we say, but some of it is obviously based on what Sarah was showing us, but, um, but, you know, there’s a lot that we need to learn from the past and predict the future, so that we can try not to repeat it the same way I think. Florence Hudson: Does that answer your question. Diane Goldenberg-Hart (CNI): Yes, thank you very much. I think Clifford would like to ask a question out. So I’ll hand it over to him. Cliff Lynch: Yeah, just jump in. While everybody’s thinking about questions. Um, first off, it’s wonderful that you’ve made that connection into the code data go fair thing, I think, Cliff Lynch: I did my eye on that meeting.

I think it’s going to be very timely, particularly in the kind of context. You’re putting it in the question I had. And I asked this a little timidly, not wanting to Cliff Lynch: Create any problems is Cliff Lynch: I noticed on a couple of your of the slides you had allusions to Cliff Lynch: Various NIH programs as well. And clearly in it. Well, NSF is one of the major funders in this area in the US NIH, certainly as well. Cliff Lynch: Can you speak a little bit about how you’re coordinating with or thinking about, you know, how what you’re doing connects and relates to the funding that NIH is doing in this area. Florence Hudson: Absolutely. And as RPI Jeanette wing likes to say, if you think of Kobe.

You think of NIH, because that’s where all the medical and healthcare professionals are 56:40 - Florence Hudson: So we, we really want to partner with them when we first got this award. It was so much to do in a short period of time, we had to keep our head down and just do the code rapid grants, which is what we were told to do. Florence Hudson: And now we’re going more broad across NSF, but one of our next goals is to think about how we can Florence Hudson: Work with NIH maybe do eat, you know, because even Sarah was talking about the aura and I’ll work that you know that she’s working with them. Florence Hudson: And the HP the code HP see consortium works with a lot of those different groups. So by collaborating with them are hoping we get into their projects.

57:12 - Florence Hudson: And we actually have a CIT leader Carl Bragg north and I spent some time. Last week crawling the NIH website and looking at their search mechanisms and they don’t have an API. But they do have Florence Hudson: Award databases that they download and they update every week. And so now that we’re done with all getting this all out the door. And our last year, October 30 was our last refresh within UPI database.

57:35 - Florence Hudson: Now we’re going to start looking at how can we bring some of that other data in Florence Hudson: You know, in the open public information should be fine. But we do want to open more doors into NIH to start talking about what could we do together. Florence Hudson: Especially if we’re thinking about, you know, crawling and searching data and metadata. Florence Hudson: You know, open science data, maybe not HIPAA compliant data, you know, we have to decide what data we’re talking about Florence Hudson: But I think it would be wonderful to do that more with them. So we do plan on talking more to NIH NSF has offered to connect us Florence Hudson: And I have some context, they’re actually tomorrow, we have the the computation approaches for cancer workshop.

I’m on the program committee for at SC virtually 58:12 - Florence Hudson: And that’s working with NCI. So we are, we do plan on talking more with them because we think we can help these collaborative these researchers collaborate more and make more progress. So thank you for asking that cliff. Florence Hudson: And any friends and family of NIH that wants to talk to us about this. I’m very we’re very open and collaborative, as you can tell. Cliff Lynch: I’m sure you’ve met Patty Brennan it in LM at some point.

58:37 - Florence Hudson: I haven’t, but somebody mentioned, we should talk to and Cliff Lynch: You should definitely talk to Patricia printing the LM she I think she’d be thrilled to talk with you about this. Florence Hudson: And if you would like to create, you know, like a little letter. If she knows he knows you could send an email. I don’t know if you can. I don’t want to put you on the spot, but that would be Cliff Lynch: Happy to do that. Florence Hudson: That would be great. Thank you very much, very thoughtful of you. I’d love to get that. Now we’re ready to start talking, you know, past like the heads down, you know, let’s get this done approach. So this is perfect timing. Thank you, Cliff. Sure. Florence Hudson: And I don’t want to take over Sarah and Nora.

I mean, Sarah, you may be working with some NIH researchers already 59:18 - Sarah Bowman: Yeah, I mean, so a number of our researchers are NIH funded and I think that I heard. I think maybe earlier this week I was on a meeting that Sarah Bowman: Was indicating that part of why a lot of even the more bio medically relevant funding for the rapids went through NSF was in part because Sarah Bowman: The NSF is really equipped to be able to provide funding that quickly. With the rapid award and at the NIH didn’t necessarily have have those mechanisms in place and so Sarah Bowman: I do think that there’s going to be more NIH funding kind of available and and so, but it’s just a longer process for NIH so Florence Hudson: Very interesting. Thank you for sharing that. Sarah Bowman: Yeah, and I may know some people who might be interested in. So I’ll touch base with you kind of offline. So Florence Hudson: Thank you, Sarah, that would be great. Yeah, all friends and family.

Welcome, you know, new friends are making a lot of new friends with this whole effort so that would be wonderful. Thank you. Florence Hudson: Any other comments or questions, Jeremiah, did you want to add anything to that. Florence Hudson: And he may be Florence Hudson: Having connectivity issues we’re all sharing connectivity internet and our computer blew up issues today so Florence Hudson: It’s really a miracle. We got this done so thank our team. Diane Goldenberg-Hart (CNI): There do seem to be goblins in the machine today. Cliff Lynch: For sure. We’re so glad you overcame them and Cliff Lynch: Joined us. This has been a wonderful presentation.

And I’m really glad that our community had an opportunity to get caught up on this great development. Florence Hudson: Well, sure. And I need to, we need to think in SF and Triton body was going to try to join us today, but he had another meeting. He’s our program officer for this and I have to thank Martin Halbert for mentioning, we should think about presenting Florence Hudson: You know, from the hubs, you know, on, on this. So I have to thank Martin as well. Cliff Lynch: Yes, Martin is here in the audience. Florence Hudson: Thank you, Martin. Cliff Lynch: Well, I’m not seeing any more questions and we are at time Florence Hudson: Right. Am I right on time. Thank you so much for having us.

And thank you, Nora Sarah and Jeremiah for your team work. We really appreciate it. Sarah Bowman: Thank you for having me. Thank you. Diane Goldenberg-Hart (CNI): Thank you. Yes, thank you, thank you all for, for coming. I’m sorry, Cliff. Go right ahead. Cliff Lynch: I was just thanking them as well. Diane Goldenberg-Hart (CNI): Great. I think we’ll do what we have been doing which is go ahead and end the recording at this point and thank all of our attendees for joining us and any attendees who want to Diane Goldenberg-Hart (CNI): Stay with us here. Diane Goldenberg-Hart (CNI): Please feel free to stay on after we turn off the recording just raise your hand. We can unmute you. And you can approach the podium and have a chat with our speakers.

And with that, I will be ending the recording and thanking you all and wishing you a lovely evening, good night. Florence Hudson: Thank you. Good night. Thank you. Nora Garza: Good night. Bye bye. .