WCET Webcast: Alternative Assessments in Digital Learning in Higher Education: Global Perspectives

00:49 - AS WE GO THROUGH, IF YOU HAVE ANY QUESTIONS, ENTER 14:01:11 THEM INTO THE Q&A, AND WE’LL GET THOME THEM DURING 14:01:14 THE Q&A PORTION. IF THERE’S A QUESTION WE FEEL LIKE WE NEED ADDRESS, ON 14:01:17 OUR MODERATOR WILL JUMP IN AND HANDLE THAT.

01:01 - WE’LL POST A LINK TO THE SLIDES SO YOU CAN DOWNLOAD THAT AND FOLLOW 14:01:24 ALONG. AND WE ARE RECORDING THIS AND WILL MAKE IT AVAILABLE TO 14:01:27 YOU AS SOON AS WE CAN. CAPTIONS ARE ALSO AVAILABLE, IF 14:01:30 YOU’D LIKE TO ACCESS THOSE. AND 14:01:34 WE TEND TO HAVE A PRETTY ACTIVE TWITTER BACK CHANNEL. THAT 14:01:38 HASH TAG IS 14:01:41 #WCETWEBCAST. THANK YOU SO MUCH TO OUR 14:01:45 PARTNER ONE HE. THEY’VE INTRODUCED US TO OUR PANELISTS 14:01:48 SO WE’RE GRATEFUL 14:01:52 FOR THE PARTNERSHIP.

VITAC IS OUR PARTNER 14:01:56 FOR CAPTIONS. ANY QUESTIONS, ENTER THEM INTO THE Q&A.

01:39 - IF THEY GET PUT IN CHAT, WE OFTEN LOSE THEM. 14:02:01 WE WANT TO MAKE SURE WE CAN RANGLE ALL THE QUESTIONS VIA 14:02:06 THE Q&A. WE HAVE A WONDERFUL MODERATOR TODAY. SHE’S ON THE WCET STEERING 14:02:09 COMMITTEE AND IS A CHERISHED 14:02:14 MEMBER. THE INTERIM VICE PRESIDENT FOR STRATEGIC 14:02:18 INITIATIVES AT THE NATIONAL DIVISION AT 14:02:22 RIO SALADO COLLEGE. SHE HAS OVER 20 YEARS EXPERIENCE IN 14:02:25 ONLINE LEARNING. SHE STARTED WHEN SHE WAS BARELY 10, I’M SURE 14:02:29 OF IT. ELIAS SERBS ON THE PRESIDENT’S 14:02:31 EXECUTIVE TEAM WHICH IS A PIONEER IN DISTANCE EDUCATION AND HAS A 14:02:35 NATIONAL AND GLOBAL REACH.

SHE PROVIDES LEADERSHIP FOR 14:02:39 STRATEGIC PLANNING, INSTITUTIONAL RESEARCH, COMPLIANCE 14:02:42 AND ACCREDITATION, CONDUCT AND COMMUNITY STANDARDS, 14:02:45 DISABILITY SERVICES, MARKETING, DEVELOPMENT, 14:02:48 STRATEGIC PARTNERS, PARTNERSHIPS AT THE NATIONAL DIVISION.

02:32 - BEFORE I HAND IT OVER TO JANELLE, I’D LIKE US 14:02:54 TO TAKE A MOMENT TO 14:02:58 REFLECT ON – WCET IS BASED IN 14:03:01 BOULDER, COLORADO. AND WE WERE FACED WITH TRAGEDY YESTERDAY. 14:03:05 SO I’D LIKE US ALL TO TAKE A MOMENT TO THINK BOSE THAT YOU LOVE AND 14:03:10 CHERISH VERY MUCH.

03:00 - >> THANK YOU. 14:03:23 NOW I’D LIKE TO GO AHEAD AND LET JANELLE TAKE IT 14:03:27 OVER. THANK YOU. >> THANK YOU, MEGAN, AND WELCOME, 14:03:30 EVERYONE. WE ARE SO EXCITED TO HAVE YOU THIS AFTERNOON. 14:03:34 SO GOOD AFTERNOON, GOOD EVENING, OR GOOD MORNING 14:03:37 DEPENDING ARE WHY YOU’RE JOINING US FROM 14:03:41 TODAY. AS A MEMBER OF THE CWET STEERING COMMITTEE, I 14:03:44 HAVE THE DISTINCT PLEASURE OF WELCOMING YOU TO THE WEBCAST AND 14:03:47 INTRODUCING YOU TO OUR PANELISTS. LET ME SAY YOU ARE IN 14:03:50 FOR A TREAT TODAY AS 14:03:54 WCET AND ONE HE HAVE LINED UP 14:03:57 PROBABLY IS EXPERTS ON ASSESSMENT.

OUR 14:04:01 PANELISTS BRING 14:04:03 EXPERTISE ADS PRACTICIANERS WHO ARE WIDELY 14:04:08 PUBLISHED IN THE FIELD. TODAY’S THREE PRESENTERS WILL FOCUS ON 14:04:12 E-PORTFOLIOS, SIMPSON ASSESSMENT, FORMATIVE ASSESSMENT, AND THE USE OF 14:04:15 EDUCATIONAL TECHNOLOGY IN ASSESSMENT. MY ROLE WILL BE TO 14:04:18 FACILITATE THE Q&A PANEL, SO PLEASE ADD YOUR QUESTIONS 14:04:21 AND USE THIS OPPORTUNITY TO ENGAGE WITH OUR 14:04:25 EXPERTS. WE WILL OPEN DISCUSSION AFTER A BRIEF 14:04:28 PRESENTATION. WITHOUT FURTHER ADO, PLEASE JOIN ME IN WELCOMING 14:04:31 OUR PANELISTS WHO WILL INTRODUCE THEMSELVES IN ORDER OF 14:04:35 APPEARANCE.

HELLO, EVERYONE. IT’S A PLEASURE TO BE WITH YOU 14:04:38 TODAY. I’M DAVID HUBERT. I’M BEEN 14:04:42 AT SALT LAKE COMMUNITY COLLEGE FOR 27 YEARS AS A 14:04:45 POLITICAL SCIENCE FACULTY MEMBER, AS A DIRECTOR OF OUR FACULTY TEACHING 14:04:48 AND LEARNING CENTER, AND NOW AS A ASSOCIATE 14:04:51 PROVOST FOR LEARNING 14:04:55 ADVANCEMENT. >> THANKS, DAVID.

04:39 - GOOD MORNING, EVERYONE. I’M FROM MELBOURNE, 14:05:02 AUSTRALIA. REALLY MY PLEASURE TO BE HERE WITH 14:05:06 YOU ALL TODAY. I WEAR TWO 14:05:09 HATS AT THE UNIVERSITY. I’M 14:05:12 ONE OF A PROFESSOR, SO IN THIS REGARD I’M A TEACHER AND 14:05:15 A RESEARCHER. I’M 14:05:18 ALSO THE CAUSE CHARACTER FOR MASTERS AND GRADUATE CERTIFICATE 14:05:23 OF OFFICIAL LEARNING LEADERSHIP AT THE UNIVERSITY, AND I WEAR THE OTHER HAT AS 14:05:26 DIRECTOR DIGITAL LEARNING FOR THE UNIVERSITY. SO I HEAD UP A TEAM OF 14:05:31 LEARNING DESIGNERS, LEARNING TECHNOLOGISTS THAT WORK ON A NUMBER OF 14:05:34 INNOVATION PROJECTS AT THE UNIVERSITY.

LOVELY TO BE HERE WITH YOU ALL 14:05:37 AGAIN. OVER TO YOU, STEVE.

05:21 - >> THANK YOU. MY NAME IS STEVE JOORDENS, I’M A 14:05:43 PROFESSOR OF PSYCHOLOGY FROM THE UNIVERSITY OF TORONTO IN CANADA 14:05:46 WHERE IT IS AFTERNOON. WE’VE SPANNED THE TIME ZONES.

05:30 - YEAH, IN ADDITION TO TEACHING PSYCHOLOGY BOTH AT U OF T AND 14:05:53 CORSAIRA. ORG, I AM ALSO DIRECTOR OF OUR ADVANCED 14:05:56 LEARNING TECHNOLOGIES LAB WHERE WE DO RESEARCH 14:05:59 ON EDUCATIONAL TECHNOLOGIES BUT ALSO CREATE SOME 14:06:03 OURSELVES. I LEAD A SORT OF PARALLEL LIFE TO CHIA WHICH 14:06:06 IS INTERESTING. WE’RE LIKE SIMILAR PEOPLE ON OPPOSITE SIDES OF THE 14:06:10 PLANET. SO I REALLY LOOK FORWARD TO PRESENTING BOTH OF DAVID AND 14:06:13 CHIA TODAY. I’M GOING TO TURN IT OVER TO DAVID TO GET THINGS 14:06:17 STARTED.

OKAY. THANK YOU, STEVE. WELL, I HAVE ABOUT TEN MINUTES, I WANT TO TELL 14:06:23 YOU THE STORY OF IT EPORTFOLIOS AT SALT LAKE COMMUNITY COLLEGE. WE’VE 14:06:27 HAD A LARGE IMPLEMENTATION OF E-PORTFOLIO IN OUR 14:06:30 GENERAL EDUCATION PROGRAM FOR 11 YEARS 14:06:33 NOW. SO WE HAVE QUITE A BIT OF EXPERIENCE WITH 14:06:37 IT. SINCE IT IS ANCHOORNED IN OUR GENERAL EDUCATION 14:06:40 PROGRAM, IT TOUCHES ALMOST ALL OF OUR STUDENTS. IT’S 14:06:43 NOT A GRADUATION REQUIREMENT, BUT IT IS A COMMON PEDAGOGY 14:06:47 THROUGHOUT OUR GENERAL EDUCATION PROGRAM.

06:31 - ALL GENERAL EDUCATION COURSES NEED TO IDENTIFY 14:06:54 AT LEAST ONE WHAT WE CALL A SIGNATURE 14:06:57 ASSIGNMENT AND HAVE STUDENTS PUT THAT IN THEIR 14:07:00 E-PORTFOLIO TO REPRESENT THE COURSE, ALONG 14:07:04 WITH AT LEAST ONE REFLECTION. NOW THE 14:07:10 SIGNATURE ASSIGNMENTS HAVE TO BE REAL-WORLD DEMONSTRATIONS OF KNOWLEDGE, SO NOT 14:07:13 QUIZZES OR TESTS, BUT TAPERS, 14:07:16 PRESENTATIONS – PAPERS, PRESENTATIONS, ART PROJECTS, 14:07:19 THINGS LIKE THAT. AND THEY HAVE TO TOUCH 14:07:22 TWO OR MORE OF THE GENERAL EDUCATION LEARNING 14:07:26 OUTCOMES.

SO CRITICAL THINKING, EFFECTIVE 14:07:29 COMMUNICATION, QUANTITATIVE LITERACY. AND THAT’S ALL UP TO THE 14:07:33 INSTRUCTOR. WE ENCOURAGE FACULTY TO 14:07:36 IDENTIFY MORE THAN ONE SIGNATURE ASSIGNMENT 14:07:39 SO THAT STUDENTS HAVE THE ABILITY TO 14:07:42 CURATE THEIR E-PORTFOLIO AND DECIDE FOR THEMSELVES WHAT 14:07:45 TO PUT IN THE PORTFOLIO FOR THEIR CLASS – FOR 14:07:49 THAT PARTICULAR CLASS. AND WHEN IT COMES TO REFLECTION, WE 14:07:52 SUGGEST THAT FACULTY ASK STUDENTS 14:07:55 A VARIETY OF REFLECTION PROMPTS AGAIN SO 14:07:58 THAT STUDENTS COULD CHOOSE WHAT TO PUT IN THEIR PORTFOLIO 14:08:02 AS THEIR FINAL REFLECTION.

07:47 - LET’S GO ON TO THE NEXT SLIDE. THIS IS HOW WE 14:08:10 CONCEPTUALIZE AN E-PORTFOLIO. IT’S A HUB 14:08:13 THAT CONTAINS LOTS OF GOODIES FROM THE POINT OF VIEW OF A STUDENT.

07:58 - SO YOU CAN SEE THE SIGNATURE ASSIGNMENTS 14:08:20 THERE AND THE REFLECTIONS. BUT 14:08:23 STUDENTS CAN ALSO PUT CO-CURRICULAR ACTIVITIES 14:08:26 IN THERE, CLUB ACTIVITIES, SPORTS, 14:08:30 STUDY ABROAD. THEY PUT THEIR GOALS IN 14:08:33 THERE, AND WE SUGGEST THAT THEY FREQUENTLY 14:08:37 UPDATE THEIR GOALS. THEY CAN DO 14:08:41 SELF-ASSESSMENT, AND THAT’S SOMETHING THAT WE’VE ALREADY TRIED TO 14:08:44 STRESS. WE’RE REALLY – BY 14:08:47 ANCHORING IN OUR GENERAL EDUCATION PROGRAM, OUR E-PORTFOLIO IS 14:08:51 DESIGNED TO HELP STUDENTS BECOME REFLECTIVE 14:08:54 PRACTITIONERS REGARDLESS OF WHAT MAJOR THEY CHOOSE.

AND THAT’S REALLY 14:08:57 ONE OF OUR GOALS. AND THEN OF COURSE 14:09:00 THE SIGNATURE ASSIGNMENTS ARE KEY TO LEARNING OUTCOMES.

08:44 - SO WE’RE HOPING THAT STUDENTS BECOME MORE INTENTIONAL ABOUT 14:09:07 ACHIEVING THOSE LEARNING OUTCOMES IN OUR 14:09:11 GENERAL EDUCATION PROGRAM. IF WE GO 14:09:14 TO THE GEX SLIDE, THIS IS WHAT WE THINK COMES OUT OF 14:09:17 AN E-PORTFOLIO. 14:09:21 EVERYTHING FROM INTENTIONALITY AS I MENTIONED, THE NOTION OF 14:09:26 BECOMING A REFLECTIVE PRACTITIONERPRACTITIONER, DEVELOPING A 14:09:29 SCHOLARLY IDENTITY. IT’S A TOOL FOR APPLYING FOR 14:09:33 SCHOLARSHIPS AND TRANSFER. IT’S A SHOWCASE 14:09:36 OF STUDENTS ENGAGEMENT IN OUR 14:09:39 GENERAL EDUCATION PROGRAM.

IT DEMONSTRATES 14:09:42 EDUCATIONAL COHERENCE AND THE UNIQUE JOURNEY THAT EACH 14:09:45 STUDENT HAS TAKEN THROUGH OUR 14:09:49 GENERAL EDUCATION PROGRAM. NOW LET ME PAUSE HERE FOR A MINUTE AND GO OVER A LITTLE 14:09:53 BIT OF HISTORY. BACK IN 2004, A 14:09:57 PREDECESSOR IN MY POSITION AND A NUMBER OF FACULTY, 14:10:00 AND I WAS ONE OF THOSE FACULTY, WE WERE QUITE CONCERNED 14:10:04 THAT OUR GENERAL EDUCATION PROGRAM BECAUSE IT WAS MENU 14:10:07 DRIVEN, STUDENTS WERE EXPERIENCING IT AS 14:10:10 ONE UNRELATED COURSE AFTER ANOTHER THAT THEY’D FIGURE 14:10:13 THEY HAD TO TAKE BEFORE THEY COULD GET INTO THEIR MAJOR.

AND 14:10:16 THERE WAS NO REAL COHESION, NO INTERGRATION 14:10:20 THERE. AND WE WERE QUITE CONCERNED ABOUT THAT.

10:04 - AT THE SAME TIME, WE HAD A VISIT FROM 14:10:26 OUR REGIONAL ACCREDITOR WHO WASN’T 14:10:29 CONCERNED SO MUCH ABOUT THAT AS AN ISSUE, BUT THEY WERE 14:10:33 CONCERNED THAT AT THE TIME THE COLLEGE WAS NOT 14:10:36 ASSESSING ITS GENERAL EDUCATION PROGRAM IN ANY 14:10:39 MEANINGFUL WAY. SO WE SORT OF PIGGY BACKED THOSE TWO 14:10:43 PROBLEMS TOGETHER. THE LACK OF INTEGRATION AND COHERENCE 14:10:46 IN OUR GENERAL EDUCATION PROGRAM WITH THE FACT THAT WE WEREN’T 14:10:50 ASSESSING IT. AND AT THAT TIME, WE RECEIVED A 14:10:53 SMALL GRANT, AND WE STARTED PILOTING 14:10:56 E-PORTFOLIOS.

I WAS IN THE FACULTY COHORT THAT STARTED 14:11:00 PILOTING E-PORTFOLIOS IN OUR CLASSES. AND 14:11:03 AFTER THREE YEARS, WE DECIDED TO PUT FORTH 14:11:07 A PROPOSAL TO MARRY ALL 14:11:10 OF THIS TOGETHER AND USE E-PORTFOLIO AS A COMMON 14:11:13 PEDAGOGY IN ALL GENERAL EDUCATION COURSES WITH THIS IDEA OF 14:11:16 SIGNATURE ASSIGNMENTS AND REFLECTION. A 14:11:19 BECAUSE IT INTEGRATES AND MAKES INTENTIONAL 14:11:23 OUR GENERAL EDUCATION PROGRAM, AND B, IT PROVIDES 14:11:26 US A PLATFORM TO ASSESS THE 14:11:29 PROGRAM.

AND SO WE DID THAT, AND WHEN OUR 14:11:32 CREDITOR CAME BACK TEN YEARS LATER, WE ACTUALLY 14:11:36 RECEIVED A COMMENDATION FOR THE WAY THAT WE ASSESS 14:11:39 OUR GENERAL EDUCATION PROGRAM. AND I MUST SAY THAT 14:11:43 WE REALLY HAVE A – A 14:11:46 TREMENDOUS INSIGHT INTO HOW STUDENTS ARE 14:11:49 ACTUALLY EXPERIENCING GENERAL EDUCATION. YOU THINK 14:11:52 YOU KNOW WHEN YOU DESIGN A PROGRAM HOW STUDENTS 14:11:56 EXPERIENCE IT BECAUSE YOU’VE CONTROLLED ALL THE INPUTS, RIGHT.

11:40 - BUT UNTIL YOU CAN SEE STUDENTS 14:12:02 DEMONSTRATE THEIR LEARNING WITH SIGNATURE ASSIGNMENTS AND 14:12:05 THEIR REFLECTION, YOU DON’T REALLY HAVE A FULL 14:12:08 UNDERSTANDING. LET’S GO TO THE FINAL 14:12:12 SLIDE NOW. SO WE’RE VERY INTENTIONAL ABOUT CLOSING THE LOOP ON 14:12:16 ASSESSMENT. IF WE START AT THE TOP 14:12:19 THERE, EVERY YEAR INSTITUTIONAL RESEARCH GRABS A SAMPLE 14:12:22 OF GRADUATING STUDENTS. THESE ARE STUDENTS 14:12:25 WHO COMPLETED ALL OF THEIR GENERAL EDUCATION PROGRAM WITH 14:12:28 US.

SO WE WEED OUT THE STUDENTS WHO TRANSFERRED IN 14:12:32 FOR ASSESSMENT PURPOSES. AND THEN OUR 14:12:35 E-PORTFOLIO OFFICE ORGANIZES AN 14:12:38 ASSESSMENT OF THESE E-PORTFOLIOS, AND WE HAVE 14:12:41 CREATED A LARGE 14:12:45 RUBRIC THAT’S BASICALLY WE HAVE COBBLED 14:12:48 TOGETHER NEW 14:12:51 VALUE RUBE RISKS, RUBRICS WE’VE DEVELOPED 14:12:54 ON OUR OWN, OTHERS THAT HAVE BEEN MODIFIED. AND 14:12:57 THEN WE HAVE TEAMS OF FACULTY WHO ARE 14:13:00 ASSIGNED A PORTION OF THE – ASSIGNED 14:13:04 CERTAIN LEARNING OUTCOMES.

EACH TEAM DOES A SESSION, AND 14:13:07 THEN THEY REVIEW ALL OF THE E-PORTFOLIOS, AND 14:13:10 THE E-PORTFOLIO OFFICE WRITES A REPORT WHICH IS SUBMITTED 14:13:13 TO THE COLLEGE COMMUNITY, PARTICULARLY THE GENERAL 14:13:16 EDUCATION COMMITTEE AND THE ACADEMIC DEPARTMENTS.

13:00 - AND THROUGH THAT, WE HAVE 14:13:23 DEMONSTRABLY IMPROVED TEACHING AND LEARNING IN A NUMBER OF SECTIONS OF 14:13:26 OUR GENERAL EDUCATION PROGRAM. AND THEN EACH YEAR WE DO THIS, 14:13:29 SO WE’RE GETTING AN ANNUAL LOOK AT OUR 14:13:33 OUR GENERAL EDUCATION PROGRAM AND 14:13:36 STUDENTS’ EXPERIENCE IN IT. AND THEN FINALLY, I WILL SAY 14:13:40 THAT ONE OF THE UNEXPECTED BENEFITS OF THE 14:13:43 E-PORTFOLIO IS THAT STUDENTS ARE STARTING TO 14:13:46 APPRECIATE THE ABILITY TO DO 14:13:49 SELF-ASSESSMENT.

THEY’RE SEEING THEMSELVES DEVELOP OVER 14:13:52 TIME, EVEN THOUGH THEY’RE ONLY WITH US FOR, 14:13:56 YOU KNOW, TWO YEARS, THEY CAN SEE THEIR PROGRESS 14:13:59 OVER TIME. THAT’S BEEN VERY GRATIFYING. NOW I’LL 14:14:03 TURN IT OVER. >> BEFORE WE MOVE ON, DAVID, 14:14:06 CAN YOU ACTUALLY ADDRESS THE QUESTION – WHICH 14:14:14 E-PLATFORM ARE YOU USING FROM 14:14:17 YOUR EXPERIENCE? >> WHEN WE STARTED WE WERE HAVING BUDGET DIFFICULTIES BECAUSE OF THE 14:14:21 GREAT RECESSION. WE OPTED NOT TO CHOOSE A PARTICULAR PLATFORM, AND WE 14:14:24 ALLOWED STUDENTS TO CREATE IN ANY 14:14:27 PLATFORM.

MOST COMMON ONES WERE 14:14:31 WEBLY, WORD PRESS, WIX, THINGS LIKE THAT.

14:15 - WE DEVELOPED A SYSTEM SO REGARDLESS OF THE PLATFORM WE COULD ASSOCIATE IT 14:14:37 WITH THE STUDENTS’ NAME AND THE CLASS ROLE. IT’S QUITE EASY FOR 14:14:40 THE FACULTY MEMBER TO GET TO THAT. THAT WORKED FOR A 14:14:46 WHILE. IT WELL SOME 14:14:49 SHORTCAN ASKINGS. NOW WE HAVE A PLATFORM DEVELOPED AT THE 14:14:52 RHODE ISLAND SCHOOL OF DESIGN. AND IT’S BEEN WONDERFUL FOR US. THERE 14:14:56 ARE A NUMBER OF OTHER PLATFORMS 14:14:59 OUT THERE THAT WE LOOKED AT AND THAT WERE QUITE STRONG, BUT WE 14:15:02 HAPPENED TO PICK DIGICATION.

THANK YOU 14:15:05 FOR CLARIFYING. >> 14:15:09 SURE. >> DOCTOR? >> THANK YOU VERY 14:15:12 MUCH, DAVID AND JANELLE. IT’S INTERESTING, ISN’T IT, BECAUSE LOTS 14:15:15 OF OVERLAPPING AREAS OF 14:15:18 THINKING, THINKING OF PEDAGOGY ACROSS THE PORTFOLIO AND WHAT I’M 14:15:21 GOING TO TALK ABOUT WHICH IS SELF-AND PEER 14:15:25 ASSESSMENT. FOR THE RECORD, WE LANDED ON PEBBLE PAD AS 14:15:28 AN E-PORTFOLIO TOOL FOR THE UNIVERSITY RECENTLY.

15:12 - THAT’S ANOTHER ONE THAT’S QUITE BIG IN AUSTRALIA AT THE MOMENT. 14:15:34 SO TODAY I WANTED TO TALK TO YOU ABOUT SELF AND PEER 14:15:38 ASSESSMENT AS ANOTHER FORM FOR ALTERNATIVE ASSESSMENT. AND THIS IS SOMETHING THAT 14:15:41 I HAVE WORKED OVER THE YEARS THROUGH PRACTICE AND 14:15:44 RESEARCH. SO IN GETTING US STARTED, I WOULD LIKE TO 14:15:47 FIRST AND FOREMOST ACKNOWLEDGE THE TRADITIONAL HONORS OF THE 14:15:52 LAND THAT I’M BEING FROM AS WE GATHER FOR THIS 14:15:55 WEBINAR, PHYSICALLY DISBURSE AND VIRTUALLY CONSTRUCTED.

LET US TAKE 14:15:59 A MOMENT TO EFFECT THE MEANING OF PLACE AND IN DOING 14:16:03 SO RECOGNIZE THE VARIOUS TRADITIONAL LANDS ACROSS THE 14:16:06 GLOBE THAT WE DO 14:16:09 IN BUSINESS TODAY. AND I ACKNOWLEDGE AND PAY RESPECT TO THE ELDERS 14:16:12 PAST AND PRESENT OF THE LAND THAT 14:16:16 WE WORK AND LIVE ON AND THEIR 14:16:20 ANCESTRAL SPIRIT WITH GRATITUDE AND RESPECT. IN MY CASE, IT’S THE PEOPLE OF 14:16:23 THE KULIN NATION IN THE EAST SIDE OF MELBOURNE.

16:07 - WHY DO WE CARE ABOUT SELF AND PEER ASSESSMENTS? 14:16:29 IT IS BECAUSE OF ALL THESE REASONS THAT 14:16:32 DAVID WAS ALSO TALKING ABOUT. THE IDEA OF DEVELOPING 14:16:38 TRANSFERRABLE 14:16:41 OR 21st CENTURY SKILLSETS SO THINGS LIKE CRITICAL THINKING, COMMUNICATIONS SKILLS, DIGITAL LITERACY AND 14:16:44 INFLUENCY, THESE ARE THE KIND OF THINGS THAT WE HAVE 14:16:47 THE OPPORTUNITY TO DEVELOP IN OUR STUDENTS THROUGH THIS FORM 14:16:50 OF ASSESSMENT. IN PARTICULAR, 14:16:54 THOUGH, THIS NOTION OF E 14:16:57 VALUATIVE JUDGMENT, NEXT SLIDE, IS THE 14:17:00 CAPABILITY TO MAKE DECISIONS ABOUT THE QUALITY OF 14:17:04 WORK, OF SELF AND OTHERS.

AND THIS IS REALLY IMPORTANT, 14:17:07 ESPECIALLY IN THIS INCREASINGLY DISRUPTED AND DIGITAL 14:17:11 WORLD THAT WE’RE LIVING IN.

16:55 - THIS IDEA OF CAVE ABILITY TO EVALUATE OUR 14:17:17 OWN WORK AND OTHERS IS REALLY IMPORTANT 14:17:21 BECAUSE THERE’S A WORLD FULL OF INFORMATION.

17:05 - AND TO BE ABLE TO DISSEMINATE FROM WHAT’S FAKE 14:17:27 NEWS TO THE REAL AND WHAT’S GOOD, QUALITY 14:17:30 OF WORK LOOKS LIKE IS A REALLY – REALLY IMPORTANT 14:17:33 CAPABILITY. AND SO JUST TO SET THE SCENE 14:17:36 IN TERMS OF WHAT SELF AND 14:17:40 PEER ASSESSMENT IS, I USE ASSESSMENT 14:17:44 AS A VERY BROAD TERM. THINGS LIKE JUDGMENT 14:17:48 OR EVALUATION OR REVIEW, I INCLUDE IN TERMS OF 14:17:52 ASSESSMENT AS AN UMBRELLA TERM. SO IN THE CASE OF 14:17:56 SELF-ASSESSMENT, STUDENTS JUDGE THEIR OWN WORK, MAKE DECISIONS ABOUT 14:17:59 THEIR OWN WORK AGAINST CERTAIN CRITERIA, AND 14:18:02 PEER ASSESSMENT IS SIMILAR, THAT THEY DO 14:18:05 AMONG THEMSELVES, AROUND THEIR PEERS’ WORK.

17:49 - SO MY STORY WITH SELF AND PEER ASSESSMENT IS ACTUALLY 14:18:12 TWOFOLD. WITH 14:18:15 PRACTICE AND RESEARCH, THAT’S THE NEXT SLIDE, THANK 14:18:18 YOU. AND SO I HAVE DONE 14:18:21 SOME RESEARCH WITH RESEARCH CENTER, AND THEN I WORKED ON 14:18:25 A PROJECT AT THE UNIVERSITY. SO JUST TO SPEND A 14:18:28 COUPLE OF MINUTES TALKING ABOUT THE PROJECT 14:18:31 AT THE UNIVERSITY, NEXT SLIDE, PLEASE, 14:18:34 YES, SO THIS IS SOMETHING THAT I’VE LED. I’VE BEEN AT UNIVERSITY SINCE 2015.

18:18 - AND IT WAS ALSO TO DO WITH 14:18:42 RECOGNIZING THIS VARIED AND GOOD PRACTICE AROUND SELF AND PEER 14:18:45 ASSESSMENT THAT HAS BEEN ALREADY HAPPENING AT THE 14:18:49 UNIVERSITY. IT IS NOT BY ANY MEANS A NEW TYPE OF 14:18:53 ASSESSMENT. SO THERE ARE ALREADY LOTS OF PRACTICES HAPPENING. 14:18:56 BUT ALSO, WE SET OUT TO LOOK FOR 14:18:59 TECHNOLOGICAL SOLUTIONS IN WHICH 14:19:03 WE CAN DO THIS SELF ASSESSMENT ON LINE AT SCALE. SO IT’S 14:19:06 THROUGH THIS SIX OR SO YEARS, 14:19:10 WE HAVE GONE THROUGH ITERATIONS OF TECHNOLOGIES AND 14:19:13 UNDERSTANDING PRACTICES A LOT BETTER AT DEEPER 14:19:16 LEVEL.

SO THROUGH THE REVENUES OF RESEARCH, I HAVE 14:19:19 DONE QUITE A LOT OF RESEARCH WITH 14:19:22 CRADLE, OUR RESEARCH CENTER FOR 14:19:25 RESEARCH AND ASSESSMENT AND DIGITAL 14:19:29 LEARNING AT DEADEAKIN AND HAVE 14:19:32 PUBLISHED QUITE A BIT. IT YOU LOOK AT MY RECORD ON LINE YOU WOULD PROBABLY FIND 14:19:36 PUBLISHED PIECES AROUND THAT. SO WHAT DO 14:19:39 WE FIND THROUGH THIS RESEARCH IN TERMS OF THE 14:19:43 OPPORTUNITIES WITH SELF AND PEER ASSESSMENT? HERE’S WHAT WE FOUND 14:19:46 – AND THERE ARE SEVEN KEY THINGS THAT WE 14:19:49 FOUND THROUGH THE 14:19:53 EMPIRICAL RESEARCH IN INTERVIEWING ACADEMICS AND SYNTHESIZING THE 14:19:56 EXISTING LITERATURE.

AS I SAID, IT IS MOSTLY TO DO 14:19:59 WITH THE DEVELOPMENT OF 14:20:02 TRANSFERRABLE OR 21st CENTURY SKILLS. AND 14:20:05 IT REALLY MAKES THIS ASSESSMENT TO BE 14:20:10 AUTHENTIC IN CULTIVATING STUDENTS WHO WILL BE 14:20:13 WORK READY AND LIFELONG LEARNERS. IT 14:20:17 CULTIVATES THOSE CRITICAL LEARNING OUTCOMES INTO THE 14:20:20 DIGITAL WORLD THAT THEY WOULD BE MOVING ON 14:20:23 TO. IT ACTIVATES THE LEARNING BECAUSE IT GIVES THE 14:20:26 POWER OVER TO THE STUDENTS TO BE THE 14:20:29 ASSESSORS, RATHER THAN THE TRADITIONAL THE ASSESSED, 14:20:33 STUDENTS GET TO EXERCISE THAT POWER IN 14:20:36 EVALUATING THEIR AND THEIR PEERS’ WORK.

AND IT ALSO 14:20:40 ENGAGES STUDENTS TO UNDERSTAND THE STANDARDS AND ASSESSMENT 14:20:44 CRITERIA BETTER. OFTEN TEACHERS CREATE THEIR RUBRICS BUT 14:20:47 NOT ENGAGE STUDENTS IN BISECTING THAT 14:20:50 RUBRIC OR ASSESSMENT CRITERIA. BUT BECAUSE 14:20:54 STUDENTS ACTUALLY SOME MAY NEED TO EVALUATE THEIR OWN WORK OR THEIR 14:20:57 PEERS’ WORK, THEY REALLY NEED TO UNDERSTAND WHAT THAT STANDARD MEANS TO 14:21:00 THEM. SO THAT’S A LEARNING OPPORTUNITY. AND IN 14:21:03 THINKING ABOUT THE FEEDBACK ALSO, 14:21:07 IT COULD BE TIMELY, VARIED BECAUSE THERE ARE LOTS OF DIFFERENT TYPES OF STUDENTS IN 14:21:10 A GROUP, FOR EXAMPLE, AND APPROPRIATE FEEDBACK, 14:21:14 AMONG STUDENTS.

ESPECIALLY IN THE CASE OF 14:21:17 TEAMWORK, FOR EXAMPLE, WHERE TEACHERS WON’T BE ABLE TO 14:21:21 WORK WITH THE TEAMS OR THE GROUPS IN REGULAR 14:21:24 MANNER. STUDENTS THEMSELVES ARE WELL SITUATED TO GIVE 14:21:27 EACH OTHER FEEDBACK. SO THAT’S THE NOTION OF FEEDBACK. AND 14:21:30 OF COURSE, THERE IS A LOT OF SKILL SETS TO BE 14:21:34 ABLE TO GIVE AND RECEIVE CONSTRUCTIVE 14:21:37 AND COMPASSIONATE FEEDBACK, AND THERE’S 14:21:40 ALL SORTS OF SKILL SETS THAT WE CAN GIVE A LOT 14:21:43 THROUGH SELF AND PEER ASSESSMENT.

LAST BUT NOT LEAST, AND THIS 14:21:47 NEEDS TO BE FRAMED CAREFULLY, WHEN DESIGNED WELL 14:21:50 OR WHEN DONE RIGHT, 14:21:54 THIS FORM OF ASSESSMENT, SELF-ASSESSMENT, CAN GET 14:21:57 LESS INPUT AND TIME OR RESOURCES 14:22:00 REQUIRED OF TEACHERS BECAUSE STUDENTS BECOME 14:22:03 THEIR SELF-REGULATED LEARNERS AND ASSESSORS TO 14:22:06 DO THIS EXERCISE THEMSELVES. SO IT TAKES A BIT OF 14:22:09 A PRACTICE AND TIME TO REFINE IT. BUT 14:22:13 WHEN DONE WELL, IT COULD SAVE SOME TEACHERS TIME AND 14:22:16 INPUT.

WHAT ABOUT THE OPPOSITE SIDE OF THE 14:22:23 COIN? CHALLENGES WITH SELF AND PEER ASSESSMENT? 14:22:26 THIS IS WHAT WE FOUND AS KEY THEMES. RELIABILITY AND ACCURACY OF 14:22:30 STUDENTS’ JUDGMENT SKILLS, THERE’S ANXIETY FROM 14:22:33 TEACHERS’ PERSPECTIVE IN REGARDS TO THE 14:22:37 PERCEIVED ACTIVITIES OF THE STUDENTS. IN THE DISCIPLINARY AREA, 14:22:40 STUDENTS ARE OFTEN SEEN AS NOVICES 14:22:43 OF THAT PARTICULAR SUBJECT. SO TEACHERS HAVE A 14:22:47 BIT OF ANXIETY IN PASSING OVER THAT POWER TO THE STUDENTS 14:22:50 TO DO THAT.

22:35 - SCAFFOLDING, THE EXERCISE, ACTUALLY REVEALS THAT 14:22:58 STUDENTS DO BECOME ADEQUATE AND ACCURATE 14:23:01 ASSESSORS THROUGH PRACTICE, AND ALSO IN THE 14:23:04 CASE OF TEAMWORK AGAIN, STUDENTS ARE THE 14:23:07 APPROPRIATE ASSESSORS. THERE ARE AREAS AND WAYS 14:23:10 IN WHICH WE CAN DEVELOP STUDENTS TO 14:23:13 BE GOOD ASSESSORS OF THEIR AND THEIR 14:23:16 PEERS’ WORK. POWER RELATIONS. THIS IS SOMETHING 14:23:20 THAT’S COMMON IN A GROUP TEAMWORK ENVIRONMENT, AND THERE 14:23:23 IS A DIFFERENCE BETWEEN STUDENT CAPABILITIES AND SO 14:23:27 FORTH.

SO TEACHERS NEED TO BE READY TO 14:23:31 GUIDE STUDENTS IN RESOLVING CONFLICTS WHEN THEY ARISE AND 14:23:34 CREATING THESE SAFE LEARNING 14:23:38 ENVIRONMENTS IN WHICH SELF AND PEER ASSESSMENT CAN OCCUR. LAST BUT NOT 14:23:41 LEAST, TIME AND RESOURCE CONSTRAINTS. AND THIS 14:23:44 IS NOT PARTICULAR TO SELF AND PEER ASSESSMENT. I THINK ALL 14:23:47 OUR ACADEMICS AND 14:23:51 ACADEMIA ARE OFTEN AT BEST OF TIMES UNDER THE PRESSURE OF TIME 14:23:54 AND RESOURCES. BUT WITH SOMETHING LIKE SELF AND PEER 14:23:57 ASSESSMENTS, ESPECIALLY TEACHERS DOING IT FOR THE FIRST TIME, WE NEED 14:24:00 TO ALLOCATE ENOUGH TIME AND RESOURCES TO 14:24:04 HELP TEACHERS IN DESIGNING 14:24:08 AND IMPLEMENTING SELF AND PEER ASSESSMENT WELL.

AND OF COURSE WE ARE NOW 14:24:11 SAYING JOIN US ON LINE BECAUSE 14:24:15 EVERYTHING HAPPENS THROUGH TECHNOLOGY AND TO BE 14:24:18 ABLE TO SCALE AND FACILITATE 14:24:21 THIS PROCESS OF SELF AND PEER ASSESSMENT, WE NEED TO REALLY 14:24:24 CAREFULLY THINK AND EVALUATE AND SELECT 14:24:27 TECHNOLOGIES, AS WELL. AND I THINK STEVE IS GOING TO TALK A LITTLE BIT MORE ABOUT 14:24:30 THAT. SO I’LL LEAVE IT AT THAT. SO 14:24:33 WHAT CAN YOU DO ABOUT IT? AND 14:24:36 WE’VE DONE WHAT WE CALL CRADLE 14:24:41 SUGGESTS SERIES, A ONE PAGE OF PRACTICAL AND SUCCINCT 14:24:44 STRATEGIES IN WHICH TEACHERS 14:24:47 CAN IMPLEMENT, AND HERE ARE THE SIX THINGS WE ARE 14:24:50 RECOMMENDING.

BE EXPLICIT ABOUT THE WHY 14:24:54 AND THE VALUE OF SELL&PEER ASSESSMENT 14:24:59 TO STUDENTS AND SCAFFOLD THE EXPERIENCE SO IT’S TIME 14:25:02 AND ACTUALLY ANSWERING THAT STUDENTS 14:25:06 CAN PRACTICE THEIR SKILL IN DOING SELF AND 14:25:09 PEER ASSESSMENT. CONSULT WIDELY. AND IT COULD BE INVOLVING 14:25:12 STUDENTS IN THE PROCESS, IN CONSTRUCTING 14:25:15 RUBRIC, FOR EXAMPLE, OR TALK TO I. T.

24:59 - PROFESSIONALS IN DOING IT ON LINE. OR OTHER COLLEAGUES 14:25:22 WHO HAVE DONE THIS WELL IN THE PAST. AND REIMAGINE RESOURCES.

25:06 - AGAIN, GOING BACK TO HOW BUSY AND 14:25:30 UNDERRESOURCED OUR ACADEMICS ARE, THINK ABOUT WHERE YOU NEED TO 14:25:33 INVEST YOUR TIME. AND REALLOCATE 14:25:36 TIME AND RESOURCES 14:25:39 ACCORDINGLY. AND THIS STARTED SMALL AND SIMPLE IS MY FAVORITE. 14:25:42 AND I’M A BIG FAN OF JAMES LANGE’S BOOK “SMALL 14:25:46 TEACHING,” BUT IF YOU ARE 14:25:49 DOING SOMETHING FOR THE FIRST TIME 14:25:54 ESPECIALLY, TAKE UNIT THAT’S SMALLER COHORT IF POSSIBLE. OR PICK A FORMATIVE 14:25:57 ASSESSMENT RATHER THAN HIGH STAKE, BIG 14:26:01 EXAM TIME ASSESSMENT TO BE SELF AND PEER ASSESSMENT.

SO START SMALL AND SIMPLE. AND WITH THE VIEW OF ITERATIONS 14:26:07 FOR LONG-TERM BENEFITS. THINGS CAN GET REFINED 14:26:11 WITH ANY EDUCATIONAL ENDEAVORS THAT YOU DO OVER TIME 14:26:14 AND MULTIPLE TIMES. FINALLY, THIS IS A 14:26:17 BIT OF A SELF-PROMOTING PLUG – I 14:26:21 HAVE DONE 14:26:24 THIS WORK WITH JUAN ONE HE AND CREATED THE MICROLEARNING 14:26:27 FOR TEACHERS, AND IT’S 20 MINUTES 14:26:30 COURSE THAT PACKS OUR RESEARCH AND RECOMMENDING 14:26:33 SOME STRATEGIES THAT ARE PRACTICAL AND EASY 14:26:37 TO IMPLEMENT IN DESIGNING AN ASSESSMENT, SELF 14:26:40 AND PEER ASSESSMENT, FOR A DIGITAL 14:26:43 WORLD.

SO THANK YOU VERY MUCH FOR LENDING YOUR EARS TO LISTEN 14:26:46 TO ME. AND I’LL PASS OVER 14:26:49 TO STEVE. >> EXCELLENT. THANK YOU, BOTH DAVID AND 14:26:52 CHIE. AND LET ME START BY SAYING TO ALL 14:26:56 THE WCET FOLKS THERE IN 14:26:59 BOULDER, WE ARE ALL THINKING ABOUT YOU, ALL GRIEVING WITH YOU. IT’S 14:27:03 HORRIBLE. OKAY. I’M GOING TO LARGELY TAKE OVER SORT OF 14:27:06 WHERE CHIE LEFT OFF A LITTLE BIT. AGAIN, WE ARE BOTH 14:27:09 BIG BELIEVERS IN THIS WHAT WE CALL FORMATIVE PEER ASSESSMENT PROCESS.

26:54 - I WANT TO HIGHLIGHT THIS POINT FIRST. ONE OF THE REASONS I’M SO PASSIONATE ABOUT THIS IS BECAUSE THESE 14:27:19 ALTERNATE – THIS ALTERNATE FORM OF ASSESSMENT NOT ONLY 14:27:22 DEEPENS STUDENTS’ KNOWLEDGE BY HAVING THEM WORK WITH WHAT THEY’RE 14:27:25 LEARNING, BUT THEY ALSO ENGAGE IN 14:27:29 QUALITY-BASED DISCRIMINATION AS CHIE WAS TALKING ABOUT, TRYING TO FIGURE OUT 14:27:32 WHAT’S BETTER THAN WHAT AND WHY. THEY EXERCISE THEIR CRITICAL THINKING, 14:27:35 THEY EXERCISE THEIR CREATIVE THINKING, THEY EXERCISE THE 14:27:38 COMMUNICATION SKILLS.

AND SKILLS ONLY DEVELOP WITH USE. SO 14:27:42 THEY HAVE TO ACTUALLY USE THEM IN ORDER TO GET 14:27:45 BETTER. AND PREFERABLY WE WANT THEM TO GET A VERY 14:27:48 STRUCTURED EXPERIENCE AND A LOT OF REPETITION. YOU KNOW, THINK ABOUT LEARNING 14:27:51 IN THE ATHLETIC SPORT OR A 14:27:54 MUSICAL INSTRUMENT. YOU GET BETTER BY 14:27:59 HAVING REPEATED STRUCTURED PRACTICE. ONE OF THE EXCITING THINGS ABOUT 14:28:02 FORMATIVE PEER ASSESSMENT AS AN ALTERNATIVE ASSESSMENT STRATEGY IS IT 14:28:05 ALLOWS YOU TO GIVE STUDENTS THAT PRACTICE IN 14:28:08 WAYS THAT FIT WITHIN THE CONSTRAINTS OF OUR 14:28:11 EDUCATIONAL REALITIES.

SO YEAH, WE’LL GO TO THE 14:28:15 NEXT SLIDE THERE, SORRY, TRYING TO ADVANCE MYSELF. SO LET ME 14:28:18 – I’M GOING TO FOCUS ON NOW THIS ROLE THAT TECHNOLOGY 14:28:21 CAN PLAY IN TERMS OF SORT OF 14:28:25 AMPLING UP THE PED GOING CAL PERSPECTIVE. I’M GOING TO TALK 14:28:28 ABOUT PEER SCHRAG, A TECHNOLOGY THAT CAME OUT OF OUR 14:28:31 LAB. THE TECHNOLOGY I KNOW BEST. SO 14:28:35 I’LL BE TALKING ABOUT IT. IF YOU WANT MORE, PEERSCHOLAR. COM WILL GIVE YOU MORE 14:28:38 INFORMATION. ALSO BE AWARE THAT THERE ARE QUITE A FEW PRODUCTS OUT 14:28:41 THERE.

IT IS STILL A MATURING MARKET, BUT YOU HAVE A 14:28:44 LOT OF CHOICE WHEN IT COMES TO PEER ASSESSMENT 14:28:49 TECHNOLOGIES. YOU CAN CHOOSE THE BEST, OURS 14:28:53 OR SOMETHING ELSE. MOVING TO THE NIX 14:28:56 SLIDE TO HYPE WHY THE TECHNOLOGY. I WANT TO START BY BEING 14:28:59 SUPER CLEAR BECAUSE SOMETIMES WHEN WE TALK ABOUT THINGS 14:29:02 IT’S A LITTLE AMBIGUOUS. THE SORT OF 14:29:05 PROCESS THAT CHIE AND I WERE TALKING ABOUT AND THAT CAN FEED 14:29:08 INTO THE DIGITAL PORTFOLIOS THAT DAVID HAS TALKED ABOUT KIND OF GOES LIKE THIS 14:29:12 – JUST TO BE EXPLICIT – USUALLY WE ASK STUDENTS ON 14:29:15 CREATE SOMETHING.

AND THIS IS THE PRESSER HAS 14:29:18 EXPERTISE IN THIS. SOMETHING THAT THEY THINK IS A GOOD ACTIVITY TO HAVE STUDENTS 14:29:22 WORKING WITH THE KNOWLEDGE. BUT THEN THERE’S THESE CRITICAL SECOND 14:29:25 AND THIRD STEPS. THE SECOND STEP IS WHAT CHIE’S 14:29:28 BEEN FOCUSING ON WHEN WE ASK STUDENTS TO LOOK AT PEER 14:29:31 WORK AND TO GIVE THEM SOME FEEDBACK ON THEIR 14:29:34 WORK, PREFERABLY SOME IDEAS ABOUT HOW THEIR WORK CAN BE 14:29:38 IMPROVED. AND THIS ENGAGES A LOT OF CRITICAL THOUGHT, CREATIVE 14:29:41 THOUGHT, COMMUNICATION, AT THAT STEP.

BUT THEN THERE’S A THIRD STEP, 14:29:44 TOO, WHICH FOR ME IS ALMOST THE REAL GOAL OF ALL OF 14:29:47 THIS. THIS IS A STEP WHERE STUDENTS 14:29:51 RECEIVED FEEDBACK FROM THEIR PEERS, AS CHIE SAID, IT’S NOT ALWAYS 14:29:54 GREAT, NOT ALWAYS ACCURATE, I THINK THAT’S A GOOD THING BECAUSE WE TELL STUDENTS, 14:29:57 YOU KNOW, YOUR PEERS, SOMETIMES THEY KNOW WHAT THEY’RE TALKING 14:30:00 ABOUT, SOMETIMES NOT. YOU DON’T HAVE TO DO WHAT THEY TELL YOU TO DO. 14:30:03 WHAT YOU HAVE TO DO IS THINK ABOUT WHAT THEY TELL YOU TO DO.

29:47 - AND SO WE’RE TRYING TO ENCOURAGE THAT GROWTH MINDSET. 14:30:09 IT TURNS OUT IT’S NOT NATURAL TO HAVE A GROWTH MINDSET, 14:30:12 SO WE CAN REALLY SCAFFOLD THAT PROCESS, TEACH THEM HOW TO ANALYZE 14:30:16 FEEDBACK, AND THEN HOW TO USE IT WELL TO MAKE THE 14:30:19 WORK BETTER. AND SO THESE TWO 14:30:22 STEPS ARE DRIVEN COMPLETELY BY THE STUDENTS. ACTIVE LEARNING 14:30:25 AT ITS BEST, ASSESSMENT AS A LEARNING AT ITS BEST.

30:10 - AND THE REALLY COOL THING IS STUDENTS LIKE IT AND VALUE THE 14:30:32 EXPERIENCE. THEY WANT MORE OF IT. SO IT’S A VERY 14:30:35 POWERFUL IN THAT EDUCATIONAL LEVEL EVEN WHEN IT’S DONE IN A SORT OF 14:30:38 TRADITIONAL WAY. BUT IF WE USE TECHNOLOGY TO 14:30:41 MANAGE THIS PROCESS, THERE ARE CERTAIN OTHER ADVANTAGES, THERE’S ONE THAT I DON’T EVEN 14:30:44 HAVE LISTED HERE THAT I SHOULD HAVE WHICH IS ANONYMITY.

30:29 - WE CAN HAVE STUDENTS GIVING AND RECEIVING FEEDBACK WITHOUT 14:30:51 KNOWING WHO THE – WHO IS THE GIVER AND WHO IS THE 14:30:54 RECEIVER, AND YOU KNOW, WITH ONE FELL SWOOP, THAT ELIMINATES 14:30:57 ALL SORTS OF BIASES THAT COULD BE BASED ON GENDER 14:31:02 OR CULTURE. ALL YOU HAVE IS THE STUDENT’S WORK TO REACT 14:31:04 TO. AND SO THAT’S EXTREMELY POWERFUL IN AND OF ITSELF. 14:31:07 BUT THE REALLY NICE THING IS, YOU KNOW, IT’S GREAT TO ASK 14:31:10 A STUDENT TO ASSESS ONE PEER’S WORK, BUT I 14:31:13 LIKE TO ASK THEM TO ASSESS FIVE OR SIX PEERS’ WORK.

30:57 - THAT REPETITION OF SEEING MULTIPLE EXEMPLARS AND 14:31:19 BEING ABLE TO COMPARE THEM AND NOT ONLY COMPARE THEM WITH EACH OTHER BUT 14:31:22 COMPARE THEM WITH YOUR OWN WORK IS 14:31:27 A REALLY STRONG METACOGNITIVE CONTEXT FOR UNDERSTANDING WHERE YOUR WORK 14:31:30 FITS. AND FOR ALSO GIVING YOU THIS REPEATED STRUCTURED 14:31:33 PRACTICE ENGAGING THOSE SKILLS. SO THE 14:31:36 TECHNOLOGY CAN HANDLE ALL THE ASSIGNING FIVE 14:31:39 PEERS, THEN BRING IT BACK, FIGURE OUT WHAT GOES WITH WHAT. AND 14:31:43 THAT ALLOWS US TO DO THIS PRACTICE IN A REALLY 14:31:46 POWERFUL WAY THAT CAN REALLY BUILD THE SKILLS WE WANT TO 14:31:49 BUILD.

IN ADDITION, YOU KNOW, 14:31:52 CHIE WAS TALKING ABOUT MICRO-LEARNING IN HER COURSE WITH ONE HE WHICH 14:31:55 WAS FANTASTIC, I TOOK A LOOK AT IT. WE CAN EMBED 14:31:58 MICROLEARNING RIGHT IN HERE. THIS IS ONE OF THE THINGS I LIKE TO POINT 14:32:01 OUT. OUR FACULTY HAVE A LOT OF EXPERIENCE IN THEIR SUBJECT 14:32:04 AREAS. THEY DON’T NECESSARILY HAVE A LOT OF EXPERIENCE 14:32:08 IN TERMS OF COACHING STUDENTS ON HOW TO GIVE GOOD 14:32:11 FEEDBACK, MOTIVATING THEM ON THIS DO SO, OR HOW TO LEARN FROM THE 14:32:14 FEEDBACK.

AND SO WE CAN EMBED 14:32:17 IN THESE STEPS MICROLEARNING EXPERIENCES WHERE WE EXPLAIN 14:32:20 TO THE STUDENT WHY THEY SHOULD CARE ABOUT LEARNING HOW TO GIVE 14:32:24 FEEDBACK, AND WHAT GOOD FEEDBACK ACTUALLY IS. AND 14:32:27 THEN RIGHT AFTER WE TEACH THEM, WE CAN SAY, OKAY, NOW GO AND 14:32:30 DO IT. GO THAT – WHAT YOU JUST LEARNED, USE IT.

32:14 - THAT IS A REALLY POWERFUL WAY TO TEACH 14:32:37 A SKILL. MICRO-LEARNING COMBINED WITH IMMEDIATE REPEATED 14:32:40 PRACTICE. AND WE CAN DO THAT IN BOTH PHASES HERE, 14:32:43 AND IN DOING SO WE’RE EMBEDDING PED 14:32:47 DOJ TO CAL EXPERTISE. THE INSTRUCTOR, THEY HAVE 14:32:50 TO COME UP WITH A GREAT GEOGRAPHY ACTIVITY. AND 14:32:53 THEN THE TECHNOLOGY CAN WORRY ABOUT GUIDING 14:32:57 STUDENTS THROUGH THE REST OF THE PROCESS IN THE MOST OPTIMAL AND 14:33:00 EFFECTIVE AND EVIDENCE-BASED 14:33:03 WAY.

OKAY, NEXT SLIDE. I WANT TO GIVE YOU A TASTE – 14:33:06 WE’VE BEEN TALKING SO FAR ABOUT WHAT I MIGHT CALL A CHRISTMAS 14:33:10 PRESENT. I WANT TO GO A LITTLE TOWARD CHRISTMAS FUTURE FOR 14:33:13 A MOMENT. THESE PROCESSES ARE REALLY GREAT 14:33:16 FOR DEVELOPING THESE SKILLS. BUT YOU KNOW, DAVID WAS 14:33:19 TALKING ABOUT ACKNOWLEDGING, RECOGNIZING WHAT STUDENTS 14:33:22 ARE GOOD AT. YOU CAN DO THAT IN AN EXTREMELY POWERFUL 14:33:25 AND INNOVATIVE WAY WITH THESE TECHNOLOGIES. THERE 14:33:28 ARE NOW RUBRICS, THE AMERICAN ASSOCIATION OF COLLEGES 14:33:31 AND UNIVERSITIES HAS 16 RUBRICS FOR THINGS LIKE CRITICAL 14:33:35 THINKING, CREATIVE THINKING.

THESE HAVE 14:33:38 ALL BEEN VALIDATED. WE’VE DONE RESEARCH SHOWING THAT IF YOU 14:33:41 ASK, SAY, FIVE OF MY PEERS TO SCORE MY 14:33:44 WORK, HOW MUCH CRITICAL THINKING IS IN MY WORK USING A CRITICAL 14:33:48 THINKING RUBRIC, IF WE AVERAGE THOSE FIVE PEER 14:33:51 ASSESSMENTS, IT TURNS OUT THAT THEY CORRELATE VERY 14:33:54 HIGHLY WITH TRAINED 14:33:58 AAC&U EXPERT RATINGS OF CRITICAL THINKING IN THESE PIECES. SO 14:34:01 YOU CAN ACTUALLY USE THE STUDENT AVERAGES 14:34:04 ON THE FLY TO ASSESS EVERYBODY’S LEVEL OF 14:34:08 CRITICAL THINKING, CREATIVE THINKING, WHATEVER SKILL YOU HAVE A 14:34:11 RUBRIC FOR.

STUDENTS LEARN A LOT BIAPPLYING THESE RUBE 14:34:14 RIGS, CHIE HIGHLIGHTED THAT ALREADY. BUT 14:34:17 WE CAN ALSO GET A NUMBER OUT OF IT. SO EVERY 14:34:20 STUDENT WHO GOES THROUGH, WE CAN GET A SENSE OF WHERE THEY SIT ON 14:34:23 WHATEVER SKILL, AND THEN WE CAN DO THINGS LIKE, SAY, IF YOU’RE IN THE 14:34:27 TOP 10%, YOU’RE GOING TO GET SOMETHING ON YOUR 14:34:30 DIGITAL PORTFOLIO TO RESIDES THAT. AND ULTIMATELY STUDENTS 14:34:33 CAN END UP WITH A DIGITAL PORTFOLIO THAT HIGHLIGHT ALL 14:34:38 S – HIGHLIGHTS ALL OF THE SKILLS THEY’RE DOING WELL 14:34:41 IN.

A NICE REPRESENTATION OF WHO THEY ARE AND WHAT THEY’RE GOOD 14:34:44 AT. THEY CAN USE TO GUIDE PERSONAL DEVELOPMENT 14:34:48 BUT CAN ALSO BE USED TO REPRESENT THEMSELVES TO POTENTIAL EMPLOYERS OR OTHER EDUCATIONAL 14:34:51 OPPORTUNITIES, ET CETERA. SO I THINK THAT’S REALLY COOL, THAT 14:34:55 MEASUREMENT POTENTIAL. ONE LESS SLIDE TO 14:34:58 JUMP TO. HERE’S THE OTHER COOL CHRISTMAS 14:35:02 FUTURE THING. SO THIS IS JUST MEANT TO BE A MOCK 14:35:05 CURRICULUM OF A SORT. BUT IMAGINE WE HAVE INSTRUCTORS 14:35:08 IN THESE CERTAIN COURSES WHO ARE USING THIS PEER 14:35:11 ASSESSMENT PROCESS, AND THEY USE IT IN CONJUNCTION WITH RUBRICS 14:35:14 THAT GET DIFFERENT THINGS.

SO LET’S GO UP IN THE UPPER RIGHT CORNER 14:35:17 AND SAY, OKAY, AT THE END OF OUR PROGRAM, SOMEBODY USED 14:35:21 A CRITICAL THINKING OR RUBRIC. WE NOW KNOW 14:35:24 WHAT STUDENT – HOW THE STUDENTS IN THAT COURSE SCORED 14:35:27 ON THAT. WE CAN USE SORT OF AI PATTERN ANALYSIS 14:35:31 APPROACHES TO TRY AND FIGURE OUT IF ANYTHING 14:35:34 PREDICTS WHY THAT STUDENT DID SOCIAL SECURITY THEY DID.

35:18 - AND MAYBE WE FIND THAT SOMEWHERE EARLY ON, MAYBE MATH 14:35:40 241 WHERE THE INSTRUCTOR WASN’T DOING THIS, ALL OF THE 14:35:43 STUDENTS WHO DID WELL AND N CRITICAL THINKING TOOK MATH 14:35:47 241. THERE’S SOMETHING IN THE WATER 14:35:50 IN MATH 241. THERE’S SOMETHING GOOD GOING ON THERE THAT’S 14:35:53 HAVING POSITIVE DOWNSTREAM EFFECTS. MAYBE WE COULD GO AND FIND 14:35:56 OUT WHAT THAT IS DOING IS WE MIGHT WANT MORE OF THAT IN 14:36:00 OUR PROGRAM. ONCE WE CAN MEASURE THESE SKILLS, WE CAN 14:36:03 NOW INTENTIONALLY USE THAT DATA TO 14:36:06 OPTIMIZE THE LEARNING THAT WE’RE PROVIDING AND NOT JUST THE 14:36:09 LEARNING AROUND CONTENT, BUT, YOU KNOW, REALLY THE LEARNING AROUND 14:36:13 THESE TRANSVERSAL SKILLS THAT CAN REALLY HAVE AN 14:36:16 IMPACT ON STUDENT SUCCESS, PERHAPS EVEN BEYOND THE CONTENT THAT WE 14:36:19 TEACH.

SO THAT’S WHAT I FIND REALLY EXCITING ABOUT THAT. WHERE IT IS NOW, BUT ALSO WHERE THIS IS 14:36:28 GOING. AND WE’RE CERTAINLY ON THE EDGE OF ALL OF THAT, WE’RE DOING THE RESEARCH AS 14:36:33 WELL AS INCORPORATING EVERYTHING INTO PEER SCHOLAR AS WE GO. AND TO VERY EXCITING TIME FOR THAT. 14:36:36 I THINK THAT’S – THAT’S IT FOR ME. YEAH. SO – RIGHT.

36:21 - I WAS GOING TO MENTION IF YOU WANT TO SEE SOME OF THE RESEARCH ON THIS, 14:36:43 THERE IS THE PAPER I WOULD MOST RECOMMEND. IT’S OUR MOST RECENT 14:36:46 PAPER WHERE I KIND OF TALK ABOUT A LOT OF THE HIGHLIGHTS INCLUDING THE 14:36:50 MEASUREMENT ANGLE. THAT – THAT SHOULD SAY SEARCH FOR 14:36:53 JOORDENS, I HAVE AN N BETWEEN THE N 14:36:56 AND THE S, IT WILL GET YOU TO THIS PAPER. IF YOU WANT 14:36:59 MORE INFORMATION, THAT’S A GREAT PLACE TO 14:37:02 START. EXCELLENT. >> THANKS, STEVE. 14:37:05 THAT’S GREAT.

I HAVE A FOLLOWUP QUESTION FOR YOU. WHEN 14:37:09 USING SCHOLAR, DO STUDENTS DO PEER ASSESSMENTS 14:37:12 OF INDIVIDUAL COMPONENTS OR THE ENTIRE 14:37:15 E-PORTFOLIO? >> OKAY. SO AN INTERESTING THING IS YOU 14:37:18 COULD HAVE THEM DOING ASSESSMENTS ON AN E-POERCH. I WASN’T 14:37:21 NECESSARILY THINKING THAT, I WAS THINKING THEY’RE DOING ASSESSMENTS ON 14:37:24 WHATEVER SOME COURSEWORK WAS, AND THEN BASED ON 14:37:28 THE SCORES YOU GOT FOR THE STUDENT, YOU KNOW, THEIR CRITICAL THINKING IF THEY 14:37:31 WERE IN THE TOP PERCENT OR 10% OR SOMETHING, THEY COULD 14:37:34 GO TO A DIGITAL PORTFOLIO, BE REFLECTED THERE, 14:37:37 AND ALLOW EVEN THE PERSON TO COME BACK TO THAT ARTIFACT THAT 14:37:40 CREATED IT.

BUT YOU COULD, YOU KNOW, A REALLY COOL THING 14:37:44 FOR SOMEONE TO DO EARLY ON IS TO HAVE THEM 14:37:47 ASSESS THE DIGITAL PORTFOLIOS AND TO ACTUALLY SEE, WOW, THAT 14:37:50 STUDENT HAS A REALLY RICH ONE. YOU KNOW, THAT’S THE POWER 14:37:53 OF PEER ASSESSMENT IS WHATEVER YOU WANT THEM 14:37:56 TO UNDERSTAND, YOU SHOW THEM A BUNCH OF 14:38:00 EXEMPLARS, SOME THAT ARE GOOD, SOME THAT 14:38:03 ARE BAD. THEY LEAVE WITH A CLEAR SENSE OF WHAT THE CRITERION 14:38:06 IS, WHAT THEY SHOULD BE SHOOTING FOR, AND WHERE THEY 14:38:09 SIT.

AND ALSO SOME EXEMPLARS THAT SHOW THEM, YOU KNOW, 14:38:12 THE DIRECTION TO GO IN. >> THANK YOU 14:38:15 FOR THAT. IF I MAY, I’D LIKE TO MOVE FORWARD WITH 14:38:18 A FEW QUESTIONS FROM THE Q&A 14:38:21 PANEL. I’LL OPEN THIS UP TO THE FULL PANEL. SO FEEL FREE TO 14:38:25 JUMP IN. WILL YOU EACH ADDRESS HOW YOU 14:38:28 ASSESS QUANTITATIVE SKILLS AND SHARE SOME EXAMPLES? WE’LL START 14:38:32 WITH DAVID. >> I DON’T KNOW THAT I CAN SHARE AN EXAMPLE 14:38:36 RIGHT NOW. BUT IN STUDENT 14:38:39 E-PORTFOLIOS, WE USE THE VALUE RUBRIC FOR 14:38:42 QUANTITATIVE LITERACY.

THERE ARE THREE DIMENSIONS ON IT. I CAN’T REMEMBER THEM OFF THE 14:38:45 TOP OF MY HEAD. ONE IS 14:38:48 COMMUNICATING WITH QUANTITATIVE INFORMATION, ANOTHER ONE 14:38:52 IS MANIPULATION OF QUANTITATIVE INFORMATION.

38:36 - I CAN’T REMEMBER THE THIRD ONE. BUT BASICALLY WHAT WE 14:38:58 DO IS WE HAVE A TEAM OF 14:39:02 FACULTY COMB THROUGH A STUDENT’S E-PORTFOLIO 14:39:05 FOR ANY ASSIGNMENTS IN WHICH THE STUDENT MINUTES 14:39:09 LATED QUANTITATIVE INFORMATION OR USED QUANTITATIVE INFORMATION TO 14:39:12 MAKE A POINT. SO THEY IDENTIFY 14:39:16 THOSE AS OUR POOL. AND THEN ANOTHER GROUP 14:39:20 OF FACULTY THEN ASSESSES THAT USING 14:39:23 THE VALUE RUBRICS. WE HAVE NOT FELT THE NEED TO 14:39:28 CHANGE OR MODIFY THE VALUE RUBRIC FOR QUANTITATIVE 14:39:31 LITERACY.

WE FIND IT TO BE VERY, VERY ON TARGET. >> THAT’S A GREAT RESOURCE.

39:15 - WE PUT IT BOTH IN THE CHAT AND ALSO IN THE 14:39:38 Q&A PANEL. CHIE, WOULD YOU LIKE TO ADD ANYTHING TO HOW YOU 14:39:41 MIGHT ASSESS QUANTITATIVE SKILLS? >> YEAH.

39:26 - INTERESTING. AND I MIGHT LINK THIS WITH THAT NOTION 14:39:48 OF SELF AND PEER ASSESSMENT AND THE KIND OF WORK THAT WE HAVE SEEN.

39:34 - INTENSIVE QUANTITATIVE FEEDBACK, SOME 14:39:56 TEACHERS AND LEARNING CONTEXT INCLUDE THAT 14:40:00 FEEDBACKING THAT IS ACTUALLY QUANTITATIVE. AND SO STUDENTS MIGHT 14:40:03 GET A SCORE OR SOMETHING WHEN THE TEAMWORK SKILL AND 14:40:06 THEY HAVE TO DISSECT THAT, WHAT THAT MEANS IN TERMS OF THE 14:40:09 RUBRIC. BUT OF COURSE WE DON’T RECOMMEND 14:40:14 THAT TO BE JUST MARKS OR QUANTITATIVE SORT OF FEEDBACK GOING BACK TO 14:40:17 STUDENTS. WHAT’S MORE IMPORTANT IS 14:40:22 THAT QUALITATIVE FEEDBACK THAT EXPLAINS WHY THESE SCORES ARE THE SCORES 14:40:25 THAT THEY’RE GIVING.

AND THAT REALLY GIVES RICH 14:40:28 LEARNING OPPORTUNITY FOR STUDENTS TO GRAPPLE WITH THAT SORT 14:40:31 OF IDEA OF FEEDBACK. SO WE DON’T OFTEN 14:40:34 JUST DEAL WITH THE MARKS OR THE QUANTITATIVE 14:40:38 FEEDBACK, BUT WE REALLY ENCOURAGE 14:40:41 TEACHERS TO EMBED THE QUALITATIVE FEEDBACK 14:40:44 IN TERMS OF SELF AND 14:40:47 PEER ASSESSMENTS. >> I THINK THE QUESTION – STEVE, LET’S 14:40:50 GO TO YOU. IT WAS MORE ABOUT ASSESSING QUANTITATIVE 14:40:53 SKILLS OF STUDENTS. SO WE CAN WRAP THAT UP, AND THEN 14:40:56 THERE’S LOTS MORE QUESTIONS FOR YOU ALL.

SURE. 14:40:59 YOU KNOW, REALLY QUICKLY, I NOT THE POINT I WANT TO MAKE IS 14:41:04 AS LONG AS YOU CAN WORD THINGS IN A SORT OF OPEN-ENDED WAY THAT GIVES STUDENTS SOME FREEDOM TO 14:41:07 ROAM, SO IF THE QUANTITATIVE 14:41:10 QUESTION IS WHAT’S TEN TIMES 31, THERE’S A RIGHT ANSWER AND 14:41:14 PEER ASSESSMENT DOESN’T DO A WHOLE LOT. IF YOU CAN DO SOMETHING LIKE HERE’S A 14:41:17 PROOF, BUT I’VE MADE FOUR MISTAKES IN THIS PROOF, COULD BE 14:41:20 WITH RESPECT TO ORDER OF OPERATIONS OR WHATEVER I DID, I WANT YOU TO LOOK 14:41:23 AT THIS PROOF, AND I WANT YOU TO AT LEAST FIND TWO OF THE ERRORS 14:41:27 THAT WERE MADE AND TO HIGHLIGHT THOSE ERRORS AND, YOU KNOW, 14:41:30 WHAT SHOULD HAVE BEEN DONE.

AND THEN WHEN THEY’RE SEEING EACH OTHER’S 14:41:33 WORK, THEY MIGHT BE SEEING ERRORS THAT THEY DIDN’T SEE, 14:41:36 SO IN SOME CASES IT’S CONFIRMING WHAT THEY SAW. IN OTHER CASES IT’S 14:41:39 CORRECTING WHAT THEY SAW. YOU KNOW, AS LONG AS STUDENTS HAVE SOME FREEDOM TO KIND OF 14:41:43 RUN WITH IT IN A CERTAIN WAY, 14:41:46 YOU KNOW, INTERPRET THIS GRAPH OR WHAT STATISTICAL ANALYSIS 14:41:49 WE’D USE FOR THIS EXPERIMENTAL DESIGN, THOSE SORTS OF 14:41:53 THINGS WORK REALLY WELL IN PEER ASSESSMENT.

AS LONG AS THERE’S NOT THE – 14:41:56 THE ANSWER ISN’T LIKE 12 OR THE – WHAT IS IT, THE END OF THE 14:41:59 UNIVERSE WAS 31 OR WHATEVER IT WAS. YOU JUST TO AVOID 14:42:02 THOSE. >> GREAT POINTS BY ALL THREE OF YOU.

41:46 - THANK YOU SO MUCH. CHIE, I’M GOING TO REDIRECT BACK TO 14:42:08 YOU FOR THIS QUESTION. WHAT STRATEGIES DO YOU ADVISE FOR 14:42:11 STUDENTS WHOSE BACKGROUND AND/OR CULTURE CREATE 14:42:14 RESISTANCE TO PEER 14:42:19 FEEDBACK? >> OH. REALLY GOOD QUESTION, AND IT’S A 14:42:22 TRICKY QUESTION, AS WELL. I THINK IT’S 14:42:25 REALLY IMPORTANT IN ANY EDUCATIONAL CONTEXT TO SET A 14:42:28 SAFE LEARNING ENVIRONMENT, RIGHT.

42:13 - AND BECAUSE THIS COULD BE QUITE NEW TO STUDENTS, AS 14:42:35 I SAID, OUR RESEARCH SHOWS THAT STUDENTS OFTEN THINK THAT THEY’RE THE 14:42:38 TRADITIONAL ASSESSED, AND THEY DON’T HAVE TO BE THE 14:42:41 ASSESSORS IN CRITICALLY EVALUATING THEIR AND 14:42:45 OTHERS’ WORK. SOMETIMES AS 14:42:48 STEVE SAID TEACHERS’ WORK. BUT THE WORLD IS 14:42:51 CHANGING. AND WE OURSELVES NEED TO BECOME 14:42:54 CRITICAL THINKERS AND EVALUATORS OF THE THINGS THAT ARE 14:42:57 HAPPENING. SO THIS IS SOMETHING THAT THEY NEED DO.

14:43:00 SO AGAIN, TALK ABOUT THE WHY OF 14:43:04 WE’RE DOING THE SELF AND PEER ASSESSMENT AND REALLY HIGHLIGHT THE BENEFITS 14:43:07 AND LEARNING OPPORTUNITIES FOR STUDENTS TO ENGAGE WITH 14:43:10 THIS. AND THEN CREATE THE SAFE LEARNING 14:43:13 ENVIRONMENTS THAT THEY CAN ALL CONTRIBUTE 14:43:16 TO. EVEN GIVEN THE DIVERSE BACKGROUNDS THAT STUDENTS 14:43:20 MIGHT COME FROM. SO OFTEN ONE OF THE 14:43:23 STRATEGIES THAT WE RECOMMEND TO TEACHERS IS THIS NOTION 14:43:26 OF TEAM CONSENT OR 14:43:30 AGREEMENT, THAT THEY AS A TEAM COME TOGETHER 14:43:34 TO CORE CONSTRUCT.

AND SO THE RULES OF ENGAGEMENT, FOR EXAMPLE, 14:43:37 HOW OFTEN DO THEY 14:43:41 MEET, WHAT SORT OF VALUES AND PHILOSOPHIES DO YOU AGREE TO HOLD 14:43:47 ON TO THROUGH THE DURATION OF THAT TEAMWORK EXERCISE. THEY COME BACK TO 14:43:52 ASSESSING THAT 14:43:55 AND RESOLVE CONFLICT. THAT SAFE LEARNING ENVIRONMENT WHERE PEOPLE COME TOGETHER TO AGREE 14:43:59 ON SOMETHING AND THEN DO THAT TEAMWORK TOGETHER. WHICH IS SOMETHING THAT WE OFTEN 14:44:03 SEE IN THE REAL WORD, WORKPLACES. YOU NEED TO HAVE 14:44:06 A CODE OF CONDUCT.

WE NEED TO HAVE AGREEMENT ON HOW AND WHY WE’RE 14:44:09 WORKING TOGETHER. SO I THINK THAT SORT OF THING DOES HELP 14:44:12 TEACHERS AND STUDENTS IN GETTING INVOLVED WITH SELF AND 14:44:15 PEER ASSESSMENT. IT’S NOT EASY, THOUGH.

44:00 - >> THANK YOU. WOULDIN LIKE TO ADD TO THAT? 14:44:23 >> I MEAN, I’LL JUST DO A REALLY QUICK – I ALWAYS FIND FOR 14:44:26 EVERYTHING WE DO TO STUDENTS, IF WE SPEND THE TIME TO SAY THIS IS WHY 14:44:29 WE’RE DOING IT, THIS IS WHAT WE HOPE YOU WILL GAIN FROM IT, AND 14:44:33 HERE’S THE RESEARCH THAT BACKS THAT 14:44:36 UP, THAT IS REALLY CRITICAL FOR EVERYTHING WE DO, ESPECIALLY IN 14:44:39 THE SITUATION WHERE YOU ARE RIGHT, THERE ARE SOME GROUPS 14:44:42 WHOSE EDUCATIONAL EXPERIENCE ISN’T REALLY CONSISTENT WITH PEER AND 14:44:45 SELF-ASSESSMENT.

AND SO FOR THEM, THIS IS A GREAT OPPORTUNITY TO 14:44:48 KIND OF BRING THEM TO THE WORK WORLD AS CHIE HAS 14:44:51 TALKED ABOUT, AND SAY, OKAY, YOU HAD YOUR PREVIOUS EDUCATIONAL EXPERIENCE, BUT 14:44:55 YOU’RE GOING TO HAVE YOUR FUTURE LIFE, AND WE’RE GOING TO TRY 14:44:58 TO PREPARE YOU TO MAKE THAT TRANSITION WELL. >> THANK YOU. 14:45:01 STEVE, SOMEONE FROM THE AUDIENCE OFFERED THE ANSWER TO LIFE, THE 14:45:05 UNIVERSE, AND EVERYTHING IS THE NUMBER 14:45:09 42. >> SEE. SEE HOW – PEER 14:45:12 ASSESSMENT WORKS.

YES. THIS IS AWESOME. 14:45:15 LEARNING. DAVID, I HAVE A QUESTION FOR YOU. IF STUDENTS ARE 14:45:18 SHARING THEIR SOLUTIONS IN THEIR PORTFOLIOS, HOW DO YOU 14:45:22 KEEP STUDENTS FROM COPYING AND, YOU KNOW, 14:45:25 SHARING, PLAGIARIZING, ET CETERA? >> 14:45:28 YEAH. SO STUDENTS ARE IN CONTROL OF THE 14:45:31 PERMISSIONS ON THEIR E-PORTFOLIOS. SO WE’VE GOT 14:45:35 IT SET SO THAT FACULTY CAN 14:45:38 SEE ANY OF THEIR STUDENTS’ E-PORTFOLIOS, BUT 14:45:41 STUDENTS CANNOT SEE EACH 14:45:44 OTHER’S E-PORTFOLIOS UNLESS THEY GIVE EXPLICIT 14:45:47 PERMISSION.

THE DEFAULT IS THAT STUDENTS CAN’T SEE EACH OTHER’S 14:46:01 E-PORTFOLIOS. THEY WOULD HAVE TO GO IN AND FLIP APE SETTING TO ALLOW. 14:46:03 AND WE DO THIS, RIGHT, SO SOMETIMES WE HAVE STUDENTS ENGAGE IN PEER ASSESSMENT OF, SAY, IT’S TYPICALLY NOT 14:46:06 A PEER ASSESSMENT OF THE HOLISTIC E-PORTFOLIO, BUT IT MIGHT BE 14:46:09 PEER ASSESSMENT OF REFLECTION, FOR INSTANCE. I DO 14:46:12 THIS ALL THE TIME. AND STUDENTS JUST GIVE EACH OTHER PERMISSION TO 14:46:16 DO THAT, AND I ALWAYS ALLOW STUDENTS TO OPT OUT OF THAT IF 14:46:19 THEY WOULD PREFER NOT 14:46:22 TO SHARE.

THANK YOU. I’M GOING 14:46:25 TO OPEN THIS QUESTION FOR THE GROUP. CAN ANY OF THE 14:46:28 PANELISTS SPEAK TO THE CONCEPT OF SINGLE-POINT 14:46:33 RUBRIC? RUBRIC? >> SINGLE-POINT RUBRIC – 14:46:40 SINGLE-POINT RUBRIC. I – >> IT’S NOT SOMETHING THAT WE OFTEN 14:46:43 USE. >> YEAH. NO, I’M NOT SURE. 14:46:47 WHEN I THINK OF A A RUBRIC I THINK OF THE PURPOSE OF A RUBRIC 14:46:50 IS TO TAKE SOMETHING LIKE SOME AM BILK BIG USE 14:46:53 THING LIKE CRITICAL THINKING AND BREAK IT DOWN INTO CONCRETE THINGS 14:46:56 THAT THE STUDENT CAN – CAN LOOK FOR.

SO WHEN I HEAR SINGLE 14:46:59 POINT IT SOUNDS LIKE YOU’RE LOOKING FOR 14:47:02 ONE THING. SO I DON’T KNOW – IT’S A NEW CONCEPT TO ME. 14:47:06 MAYBE IT’S JUST SIMPLIFYING THE ASSESSMENT TO MAKE IT AS 14:47:09 CONCRETE AS POSSIBLE SO THAT PEOPLE CAN BE – GET USED TO THE 14:47:12 PROCESS OF SEARCHING FOR THAT THING AND COMPARING THAT 14:47:16 THING. THAT’S MY BEST FLOUNDERING 14:47:19 ANSWER. >> IT SOUNDS A BIT LIKE 14:47:22 COMPETENCY BASED RUBRIC 14:47:25 PERHAPS, THAT STUDENTS GO, YES, I CAN COO DOO THAT, OR, 14:47:28 NO, I CAN’T DO THAT, AS A SINGLE CRITERION OF 14:47:31 A RUBRIC.

I’M NOT 14:47:35 SURE. BUT WE DON’T OFTEN REFER TO 14:47:38 SINGLE-POINT RUBRIC IN OUR PRACTICE, AS STEVE 14:47:42 SAYS. IT’S OFTEN RUBRIC CONSISTING OF CERTAIN SET OF 14:47:46 CRITERIA THAT WE UNPACK WHAT THAT PARTICULAR LEARNING 14:47:49 OUTCOME OR THE TASK LOOKS TO 14:47:53 EVALUATE ON. SO, YOU KNOW, DAVID, DO YOU WANT TO ADD ANYTHING TO THAT? 14:47:56 >> I DO NOT. I’M NOT FAMILIAR WITH 14:48:01 SINGLE-POINT RULES COMMITTEE – SINGLE-POINT RUBRICS, NO DO 14:48:04 I THINK THEY’RE AT USE IN MY INSTITUTION.

47:48 - I DON’T FEEL THAT I COULD CONTRIBUTE. >> THERE’S A LINK IN THE CHAT IF YOU 14:48:11 WANT TO LEARN MORE. THANK YOU, KIM, FOR OFFERING 14:48:14 THAT NOMENCLATURE TO THE GROUP.

47:58 - THERE’S SOME INFORMATION IN THERE IF PEOPLE ARE INTERESTED IN LEARNING MORE 14:48:20 ABOUT THE CONCEPTS. AND 14:48:24 NEXT QUESTION. THIS WILL BE FOR ALL OF YOU.

48:09 - DO YOU FIND FACULTY RECEPTIVE TO IMPLEMENTING 14:48:31 SELF AND PEER ASSESSMENTS IN THEIR COURSES, AND/OR 14:48:34 WHAT STRATEGIES ARE YOU USING 14:48:37 TO INCREASE THE ADOPTION OF SELF AND PEER ASSESSMENT WITH YOUR 14:48:41 FACULTY? >> SO THIS IS FOR ME, ISN’T IT, 14:48:45 JANELLE? YEAH. SO YES, 14:48:48 IN THE CASE WHERE THEY’RE ALREADY GOOD 14:48:52 FOUNDATIONS AND PRACTICES AROUND SELF AND PEER ASSESSMENT. FOR 14:48:55 EXAMPLE, IN OUR FACULTY OF SCIENCE, WHERE 14:48:59 THERE’S LOTS OF PBL, PROBLEM BASED OR 14:49:02 PROJECT-BASED LEARNING HAPPENING IN ENGINEERING, IN 14:49:05 ARCHITECTURE, FOR EXAMPLE, THERE’S A LOT OF TEAMWORK OR 14:49:08 GROUP EVALUATION HAPPENING.

SO IN THOSE 14:49:11 KIND OF CONTEXT, YES, TEACHERS ARE OFTEN 14:49:16 RECEPTIVE TO DOING SELF AND PEER ASSESSMENT. SO 14:49:21 PRACTICE-BASED DISCIPLINRY AREAS, YES, OFTEN WE SEE 14:49:24 MORE OF AN INCLINE TO DO THIS SORT OF ASSESSMENT. BUT 14:49:27 IN OTHER CASES, WHERE THERE IS A LOT 14:49:30 OF ANXIETY ABOUT PERCEIVED EXPERTISE 14:49:34 AGAIN, THERE’S LESS INCLINE TO DO THIS 14:49:37 SORT OF ASSESSMENT. BUT WE DO 14:49:40 RECOMMEND BECAUSE AS I TALKED ABOUT LOTS 14:49:43 OF LEARNING OPPORTUNITIES THAT COME WITH SELF AND PEER ASSESSMENT, THERE ARE CERTAIN 14:49:47 CONTEXTS IN WHICH THAT IS REALLY FRUITFUL 14:49:50 FOR AND PEER ASSESSMENT TO HAPPEN.

SO WE DO 14:49:53 RECOMMEND IN SOME AREAS, BUT 14:49:57 OF COURSE, THEIR EDUCATIONAL CONTEXT NEEDS TO BE THERE. ONE THING I 14:50:00 WOULD SAY, THOUGH, IS THE IDEA OF 14:50:04 TECHNOLOGY THAT HELPS AND SCAFFOLDS THE PROCESS OF SELF AND PEER 14:50:08 ASSESSMENT REALLY HELPS. SO AS I MENTIONED OVER THE SIX 14:50:11 YEARS OR SO WE LET THIS PROJECT – LED THIS PROJECT TO 14:50:14 FIND TECHNOLOGIES, AND WE HAVE 14:50:17 ENGAGED WITH QUITE A FEW. SO PEER – THE 14:50:21 TURNING IN, SPARK CLASS, ANOTHER 14:50:25 AUSTRALIAN PRODUCT, AND NOW WE LANDED ON 14:50:29 FEEDBACK FRUIT, 14:50:32 A DUTCH COMPANY FROM THE NETHERLANDS THAT IS WORKING FOR US.

BUT 14:50:35 IT’S BEEN QUITE A JOURNEY. IF YOU LOOK INTO THE MARKET 14:50:38 OF EDUTECH IN AND AROUND SELF AND 14:50:44 PEER ASSESSMENT, I WOULD SAY IT’S STILL IMMATURE. IN 14:50:48 STEVE’S CASE, THEY DEVELOPED IN HOUSE. THAT IS NICHE AND SPECIFIC TO THEIR 14:50:51 CONTEXT. SO I THINK SOMETHING THAT WORKS IN TERMS OF THE 14:50:54 TECHNOLOGY THAT CAN HELP TEACHERS AND STUDENTS 14:50:57 IN DOING THE SELF AND PEER ASSESSMENT CAN REALLY HELP.

50:41 - BUT IT’S – IT’S A CHALLENGE TO 14:51:03 FIND WHAT WORKS FOR YOU BECAUSE THERE ARE LOTS OF DIFFERENT 14:51:07 VARIATIONS OF TECHNOLOGIES, AND EVALUATING THOSE ACTUALLY DOES 14:51:11 TAKE QUITE A LOT OF EFFORT. >> IF I 14:51:13 CAN JUST – >> STEVE – >> REALLY QUICKLY. 14:51:16 YOU KNOW, ANY TIME I PRESENT THIS, AND I OFTEN LIKE 14:51:19 TO ASK PEOPLE BRING THAT PROF 14:51:22 THAT HATES IT – DOESN’T LIKE 14:51:26 IT, BRING THEM TO THE DEMO. AND I LOOK TO GET THEM TO THE POINT 14:51:29 WHERE THEY SAY, OKAY, I’M NOT GOING TO USE IT BUT I CAN UNDERSTAND 14:51:32 WHY OTHER PEOPLE WILL.

USUALLY ONCE YOU LAY OUT THE PROCESS AND EXPLAIN 14:51:35 IT TO CATHEDRALORS, THEY GET IT – EDUCATORS, THEY GET IT. 14:51:38 IT PASSES THE SNIFF TESTED, TEST, AND SHAP SAY I CAN SEE WHY THAT 14:51:42 WOULD BE A GREAT EDUCATIONAL EXPERIENCE. THE WORRY IS, OH, 14:51:45 I DON’T KNOW HOW TO TELL STUDENTS HOW TO GIVE FEEDBACK WELL. I DON’T KNOW 14:51:49 ITSLYLY 14:51:52 – DON’T KNOW HOW TO TEACH THEM TO LEARN FROM FEEDBACK. IT’S 14:51:55 IMPORTANT TO FIND SOLUTIONS WHERE EXPERTISE IS EMBEDDED 14:52:02 EVEN AS DEFAULTS SO AN INSTRUCTOR IT FIND IT’S PROBABLY PERFECTLY FINE THE 14:52:05 WAY IT IS, BUT IF I GET EXPERTISE AND WANT TO START TWEAKING WITH 14:52:10 THAT STUFF, I CAN.

I CAN USE IT OUT OF THE CAN, AND MY STUDENTS WILL HAVE A GREAT EXPERIENCE, 14:52:13 AND I WILL HAVE A GREAT EXPERIENCE. SO I THINK THAT’S PART OF WHY I 14:52:17 HIGHLIGHTED THAT OPPORTUNITY WITH TECHNOLOGY TO EMBED THE 14:52:21 MICROLEARNING IN THERE AND EMBED THE EXPERTISE. SOME 14:52:24 OF THE NEW PEDAGOGIES DO PUSH THE 14:52:27 ENVELOPE ON RESEARCH AND NOT ALL OF OUR FACULTY REON THE 14:52:30 CUTTING EDGE OF THAT ENVELOPE. BRINGING SOME OF THAT AND EMBEDDING IT IN 14:52:33 THE PRODUCT REALLY MAKES IT THAT MUCH 14:52:37 RICHER.

I’LL 14:52:40 JUST ADD THAT PEER ASSESSMENT HAS BEEN 14:52:43 USED AT MY INSTITUTION FOR A LONG TIME. SO IT 14:52:46 PREDATES E-PORTFOLIO, FACULTY GENERALLY 14:52:49 ARE QUITE COMFORTABLE WITH IT. SELF ASSESSMENT 14:52:53 IS RELATIVELY NEW, AND E-PORTFOLIO 14:52:56 HAS REALLY HELPED TO JUMP-START 14:52:59 SELF ASSESSMENT. WE HAVE FACULTY, 14:53:02 FOR INSTANCE, WHO USE THE KIND OF I. T. RATIVE 14:53:06 PEDAGOGY THAT STEVE WAS TALKING ABOUT WHERE YOU’RE 14:53:09 GIVING THEM REPEATED PRACTICE 14:53:12 WITH THE SAME TYPE OF ASSIGNMENT OVER TIME, 14:53:15 TYPICALLY WITH DIFFERENT CONTENT, AND SO WE HAVE FACULTY 14:53:19 THAT DO THAT OVER THE COURSE OF A SEMESTER, AND 14:53:23 THEN HAVE STUDENTS COMPARE THEIR LATER WORK WITH THE 14:53:26 WORK THEY DID AT THE BEGINNING OF THE SEMESTER TO SEE THE KIND 14:53:29 OF PROGRESS THEY MADE.

AND THAT’S 14:53:33 EXCITING. >> GOOD EXAMPLES. I’M GOING TO HAVE A 14:53:36 TIME CHECK. WE HAVE TIME FOR TWO MORE QUESTIONS. I’LL OPEN THIS ONE 14:53:39 UP TO THE PANEL. CAN ANY OF THE PANELISTS HAVE 14:53:43 CONCERNS ABOUT OVER EMPHASIS 14:53:46 ON DATA AND SCORES, OVER EMPHASIZING THE 14:53:49 QUANTITATIVE I THINK CHIE ADDRESSED THIS EARLIER, TO 14:53:52 THE QUALITATIVE, OR ARE 14:53:55 THE QUALITATIVE ASPECTS INCLUDED INTENTION LEAP ON THE RUBRICS AND 14:54:00 ASSESSMENTS? HOW DO YOU ACHIEVE THAT BALANCE? >> YEAH.

14:54:03 YEAH. SO I MEAN, THE – I SORT OF HAVE TWO THOUGHTS IN MIND. 14:54:06 FIRST OF ALL, I KNOW IN THE NETHERLANDS WHERE THEY’RE DOING A LOT OF WORK EVEN AT THE 14:54:09 SCHOOL SYSTEM OF HAVING A STREAM FOR MORE ARTS KIND OF THING, 14:54:12 WHERE QUALITATIVE FEEDBACK MAKES A WHOLE LOT MORE SENSE THAN 14:54:15 QUANTITATIVE – A DANCE DOESN’T GET AN 8. 5, YOU KNOW, YOU CAN JUST 14:54:19 TALK ABOUT LOU IT MADE YOU FEEL OR THINGS LIKE THAT.

54:03 - SO YOU KNOW, IT ALLOWS THAT SORT OF RICH QUALITATIVE ASSESSMENT.

54:06 - BUT ALSO WHEN WE GET INTO THE RUBRICS AND THE USE OF 14:54:28 RUBRICS, YES, THAT WILL LEAD TO A QUANTITATIVE VALUE, BUT 14:54:31 IT’S A QUANTITATIVE VALUE THAT’S SUPPOSED TO REFLECT 14:54:35 SOMEBODY’S SUBJECTIVE ASSESSMENT OF A QUALITATIVE PIECE OF 14:54:38 WORK. AND SO WHERE I SEE IT, YOU KNOW, I LOOK AT MYSELF, AND I ALWAYS 14:54:41 USED TO SAY I WAS NEVER A GOOD MEMORIZER. WHEN YOU LOOKED AT MY 14:54:44 GPA OF MY FIRST THREE YEARS ESPECIALLY, IT IS NOTHING 14:54:47 SPECIAL. AND I WAS SORT OF B-MINUS 14:54:52 AT TOPS.

I WAS GOOD AT THINKING 14:54:55 CREATIVELY AND THAT STARTED TO PAY OFF LATER. I LIKED BEING ABLE 14:55:00 TO RECOGNIZE THESE ABILITIES IN STUDENTS AND TO BE ABLE TO SAY EVEN FROM FIRST YEAR TO RECOGNIZE, WOW, 14:55:04 THIS STUDENT IS REALLY GOOD AT ORAL PRESENTATIONS OR WHATEVER IT MAY BE. AND 14:55:07 TO GIVE THAT STUDENT SOME SENSE THAT IT’S NOT JUST ABOUT HOW MUCH 14:55:10 CONTACT CAN I STUFF IN MY HEAD AND THEN GET 14:55:14 OUT AT SOME SUMMIT OF EXAM AT THE END. IT’S A MUCH MORE 14:55:17 FORMATIVE, MUCH MORE RICH DATA-RICH, EVEN THOUGH 14:55:20 DOES HAVE, YOU KNOW, QUANTITATIVE I COULD 14:55:23 IT IS MORE QUALITATIVE AT ITS HEART 14:55:26 THAN ARE TRADITIONAL APPROACHES.

I’LL SAY THAT – I THINK 14:55:30 IT’S A GREAT QUESTION. THE WHOLE TENSION 14:55:33 BETWEEN THE QUANTITATIVE IMPERATIVE OF ASSESSMENT AND OUR SORT OF 14:55:37 HUMANISTIC, HOLISTIC INTEREST IN EDUCATION 14:55:40 AS AN INTERESTING TENSION. TO TAKE IT TO 14:55:48 ANOTHER DOMAIN, I HAVE FOUND PERSONALLY THAT IF I MEASURE MY TOMATO PLANTS IN MY GARDEN EVERY 15 MINUTES, 14:55:52 IT DOESN’T HELP THEM GROW ANY FASTER. AND 14:55:55 SO YOU KNOW, TRANSLATING THAT TO EDUCATION. ONE OF THE THINGS I 14:55:58 LIKE ABOUT THE E-PORTFOLIO IS THAT IT ALLOWS ESSENTIALLY 14:56:01 BOTH, RIGHT.

IT ALLOWS 14:56:04 YOU TO HARVEST ASSIGNMENTS IF YOU WILL AND MEASURE THEM 14:56:07 AND GIVE THEM, YOU KNOW, A QUANTITATIVE NUMBER. BUT 14:56:10 IT’S ALSO – IT SHOULD BE ANYWAY A 14:56:14 HOLISTIC REPRESENTATION OF A STUDENT’S EXPERIENCE. AND 14:56:18 SO I LIKE IT FOR THAT REASON THE WAY IT 14:56:21 KIND OF, YOU KNOW, THREADS THE LINE BETWEEN 14:56:24 THOSE TWO. >> NO, I ECHO WHAT YOU’RE SAYING, 14:56:27 DAVID. I THINK THAT HOLISTIC AS 14:56:30 A TERM IS A REALLY IMPORTANT NOTION HERE. I THINK I 14:56:33 SYMPATHIZE WITH THIS QUESTION, GREAT QUESTION.

I 14:56:36 DO FEEL A LITTLE BIT CONCERNED ABOUT THE FOCUS 14:56:39 THAT HIGHER EDUCATION ESPECIALLY PUTS ON, THE 14:56:43 NUMBERS AND QUANTITATIVE SORT OF MEASUREMENT AND 14:56:46 FEEDBACK. BUT IF YOU LOOK AT THE RUBRIC, AND THIS IS WHAT STEVE IS 14:56:50 SAYING, IT COMES WITH DESCRIPTORS. THE INNESS 14:56:53 NUMBERS ARE – THE NUMBERS ARE COMING WITH THE DESCRIPTION OF WHAT WE MEAN BY 14:56:57 THOSE NUMBERS, A FORM OF FEEDBACK. I THINK 14:57:00 WE NEED TAKE BOTH AND MORE HOLISTIC APPROACH 14:57:04 TO GIVING VARIED AND APPROPRIATE FEEDBACK.

56:48 - NUMBERS ARE JUST NUMBERS. BUT WHAT IMPACTS 14:57:11 THAT IS ACTUALLY THE LEARNING OPPORTUNITY IN 14:57:14 TALKING ABOUT CRITICALLY ENGAGING WITH 14:57:17 FEEDBACK, AND THAT’S ALSO GIVING AND RECEIVING THE FEEDBACK.

57:01 - I THINK THAT’S THE REAL OPPORTUNITY. BUT HIGHER EDUCATION 14:57:23 SECTOR, NORMALLY DOES HAVE THAT 14:57:26 FOCUS ON QUANTITY-TYPE FEEDBACK BECAUSE THAT’S A WAY THAT 14:57:29 STUDENTS PROGRESS COURSES AND GET INTO DIFFERENT 14:57:33 TYPES OF HIGHER EDUCATION PROGRAMS. SO THAT’S THE TENSION 14:57:36 THAT WE NEED TO BE MINDFUL OF. >> THANK YOU. I 14:57:39 THINK WE’LL CLOSE THE Q&A ON THAT NOTE. AND I’LL TURN 14:57:43 IT BACK OVER TO MEGAN. THANK YOU ALL FOR YOUR EXPERTISE AND 14:57:46 SHARING YOUR PRESENTATIONS TODAY.

THANK YOU. >> 14:57:48 GREAT. THANK YOU, EVERYBODY. I LEARNED SO MUCH. 14:57:51 AND THANK YOU FOR YOUR WONDERFUL QUESTIONS. I LOOK FORWARD 14:57:54 TO DIGGING THROUGH THE CHAT LITTLE MORE TO SEE WHAT MISSED IT. WE HAVE 14:57:57 JUST PULLED TOGETHER A FOLLOWUP THAT WE THOUGHT WOULD BE REALLY 14:58:01 FUN AND USEFUL. SO 14:58:05 STEVE WILL WALK US THROUGH ON APRIL 14th A DEMO 14:58:08 OF HOW THE TECHNOLOGY PLATFORM WORKS. CLICK THAT LINK. IT’S NOT ON 14:58:11 OUR WEBSITE YET BECAUSE WE LITERALLY DID JUST PULL THIS TOGETHER.

SO KUDOS 14:58:14 TO MY TEAM FOR THEIR WORK ON THAT. CLICK THAT LINK. YOU 14:58:17 CAN ACCESS THE SLIDES FROM A PREVIOUS 14:58:21 LINK, AND WE’LL BE SENDING A REMINDER OUT SO THAT YOU CAN 14:58:24 REGISTER AND JOIN US FOR THAT CONVERSATION 14:58:27 AND DEMO. AND THEN WCET MEMBERS, WE’RE 14:58:30 EXCITED THAT STEVE IS GOING 14:58:33 TO BE OUR FIRST ASK THE EXPERT IN AN ASK THE 14:58:36 EXPERT SERIES. AND YOU’LL BE ABLE TO LOG IN TO WCET 14:58:39 MIX, THE ONLINE COMMUNITY PLATFORM, AND ASK YOUR 14:58:43 QUESTIONS TO STEVE.

AND HE WILL RESPOND 14:58:46 ASYNCHRONOUSLY. LOOK FOR MORE ON THAT. THAT IS OPEN, 14:58:49 AGAIN, TO WCET MEMBERS. IF YOU’RE NEW TO 14:58:53 WCET, VISIT OUR WEBSITE, CHECK US OUT. WE HAVE LOTS OF GREAT 14:58:57 RESOURCES ON OUR SITE. WE ARCHIVE ALL OF OUR WEBCASTS, SO YOU CAN 14:59:00 SEE WHAT OTHER INTERESTING SPEAKERS AND TOPICS WE’VE HAD 14:59:04 RECENTLY. DO VISIT THE ONE 14:59:07 HE PLATFORM. THIS IS AN 14:59:11 INCREDIBLY AMAZING EMERGING NONPROFIT COMPANY FROM THE 14:59:14 UK, AND YOU CAN PSYCH UP ON THEIR BETA 14:59:17 – SIGN UP ON THEIR BETA PLATFORM AND LISTEN TO THE LEARNINGS 14:59:21 OUT THERE.

AGAIN, THE WEBCAST WAS RECORDED. WE’LL 14:59:25 SEND YOU THE LINK. IT ON OUR WEBSITE. AND WE HAVE SOME EXCITING EVENTS 14:59:28 COMING UP. OUR VIRTUAL SUMMIT WHICH IS FREE AND OPEN TO EVERYBODY, 14:59:33 KICKING OFF APRIL 6th. AND PART TWO ON MAY 4th. AND 14:59:36 WE’RE ACCEPTING PROPOSALS FOR OUR UPCOMING 14:59:39 ANNUAL MEETING NOVEMBER 2nd, AND THIS WILL BE A ONE-DAY.