As part of the work we are doing at p2pu.org we are creating a new feature for asynchronous online learning that uses game and project based learning principles; we call that a learning challenge. We have already launched a very rough beta of challenges with School of Webcraft that you can check out here. Additionally these last weeks we have been really focusing on getting our current work out there and testing it with different audiences, as well as iterating and redesigning the current model with the p2pu community in order to improve the experience.
In these past weeks we first run a bunch of workshops at the Mozilla festival “Media, Freedom and the Web” to test our existing “challenges model”. Depending on the session (we run a handful!) the participants were either novices in webmaking or expert programmers or simply people who had an interest in using the challenges model to teach about topics other than programming. During these sessions we had different types of comments, some regarding the interface and some the actual learning approach and content. Additionally during the P2PU Berlin workshop we looked more into new features that could improve the “learning challenge” experience. In this blog post I will try to sum up the feedback we got in both places; from creating a challenge, to completing the tasks, to applying for badges. Finally, we will share some of the wireframes we created in the p2pu workshop, including features the community would like to see or improve.
1. Creating a Challenge
Many participants were interested in creating their own challenges on p2pu, here is their feedback both in terms of content and user experience;
Feedback on the content
CONSTRAINTS=BETTER CHALLENGE IDEAS: we played a game during the mozilla festival were people were given constraints in order to come up with challenge ideas. They had to combine verbs, that correlated to “learning mechanics”, with artifacts, such as “video, recipe(!)”, with a word of their choice, that correlated to something they wanted to teach, for example “interactive narrative”. We found this activity to be really helpful for people to come up with ideas for challenges in a group setting.
CHALLENGE 101 in 5 STEPS: we also spend time in the p2pu workshop explaining what are challenges, when we should use them and how we can make our own challenge. During these sessions we concluded in 5 steps that one can take to create their own challenge (see picture below).
For more details on the content created for new challenges visit this etherpad.
Feedback on the UX
CREATE A CHALLENGE UX = CONFUSING: it is hard to navigate the system to create a ‘challenge’ as it is listed under study group. Many people asked “What is a “study group”? Why are we starting here?.”
ADD MORE INFORMATION: Need more copy about what a challenge is so people can understand what they are trying to do before they do it. ”Optionally provide a term more specific than study group, course, or challenge, and the corresponding description” - “We need to know what this is in advance, what is it and what is it used for, so we know what to enter here”.
NEXT STEPS IN CREATE A CHALLENGE=NOT CLEAR: Copy Issue around “Next Steps” not clear what this is doing, what are the next steps.
INPUT FIELD= LONGER : “Wish that the input field was larger for the one sentence description.”
MISSING ABILITY TO UPLOAD BADGES: there is no way to upload badges as a challenge creator, not clear what the process is to do that.
GROUP CHALLENGES: many users requested the ability to group challenges together when creating them.
ABILITY TO SYNCH CHALLENGES: instead of having challenges be asynchronous have the option to have them run for a specific time period.
2. Taking a Challenge: Many participants with various levels of expertise with programming took the challenges or tested the interface, here is their feedback both in terms of content and user experience; more detailed notes in John Britton’s etherpad here.
Feedback on Content
CREATIVE CHALLENGES vs ROTE MEMORIZATION: People thought that the HTML hunting in the world around you challenge was an inventive way of thinking about the meaning of tags; they found interactivity very motivating. “I really liked the challenge because it was fun, I like that I connected something abstract to something real (marking up the world around you)” On the other side there was a lot of controversy over the HTML by hand challenge that had learners copy code in a piece of paper. Some said things like “I didn’t do the first challenge, I hate copying things ” and for others copying HTML by hand was helpful to learn the different tags.
FINAL PRODUCT: Many participants asked for a final product that they can try out within the challenge mode. There were many ideas about this, from having a code editor embedded in the challenge to having the ability to better view blog posts or videos with peoples’ work.
PUSH FOR A CO-OPERATIVE SPACE: even though the challenges encourage some peer interaction they can be completed alone. Many people said that they would prefer to work in a group in order to complete challenges and wouldn’t have done this on their own.
INCLUDE EXAMPLES: some of the tasks are really general, it would be good to provide several examples. Maybe include pointers to external resources. “It was easy to ask for help since we are in the same place, don’t know how I would get help through the site.”
MORE BUILT IN INTERACTIVITY: include interactions such as voting for the most useful work link, battling and debating
SURPRISE: add bonus tasks and surprises for learners who might want an extra challenge.
FINDING THE RIGHT LEVEL BEFORE STARTING: add a way to find the right level challenge for before taking it. That way one avoids being stuck with a challenge that might be too easy or too hard.
PLUG AND PLAY: There were a lot of requests for more interactivity, people would like to see more happening on “the machine”, less about writing and drawing. “I want to plug and play and see it working”.
INCLUDE BLOG FEATURE: There were some requests for the p2pu platform being able to provide a blog infrastructure so users do not have to create their own blog.
COLLECTION of LINKS: “If you’ve got a bunch of people, they can see each others sites by linking to each other.” There were suggestions to create some kind of collection on the site.
TASK LINKS NOT CLEAR: Many users said that they didn’t know that they were supposed to click on the task link, so they started working on their own without using the checklist feature.
LENGTH OF CHALLENGES = CONFUSING: “Length of time for challenges seems confusing- I wish there were check boxes or scroll down menu items for this”.
CLEAR GOAL: many users agreed that there is a need to have the “challenge goal” in a clear way on the top of the challenge instead of the lengthy paragraph on the side. There were also ideas for including the “end product” as part of the “challenge goal.”
COMMUNITY FEATURES NEED TO BE MORE CONCRETE: The community features seem to be implied but not truly concrete, “how can I have more opportunities to look at others work and make new friends?”
PAIR MENTORS: users felt there should be a way to be paired with mentors who can help them complete the challenge.
3. Completing a challenge and applying for badges:
When it came to completing the challenge and applying for skill and peer badges here is what participants had to say:
Feedback about the content
MORE INFO ABOUT BADGES: It was very unclear to users, why they were getting badges and what was the relationship between these badges and open badges. Things like missing titles on the badges and concrete steps on how to get them were also an issue.
CONNECT BADGES TO TASKS:Currently you can submit a link to your work for a badge, but not for a task as part of a challenge. I would be really interested in seeing the ability to submit a link to the work you did, and have a mentor check the work and verify that the task was complete. Or if the granularity of the task is very small, then being able to submit your work at the end of a challenge to a mentor who can verify that you did the challenge. Then if you have completed the challenges in the series then you get the badge. If not a mentor, then a trusted “peer” who has “leveled up” to expert.
GETTING A BADGE WITH NO EXPLANATION: users were confused when receiving the webmaking 101 badge and Betahero badge without any explanation.
WHY SHOULD I APPLY? the interface does not explain why learner should apply for a badge. Participants made suggestions to include tips to help learners better understand why they are applying.
INTEGRATE PEER BADGES INTO DISCUSSIONS: users felt that the peer badges should be integrated in the discussion were users could award them in a simple drag and drop way.
INTRODUCE LEADERBOARDS: there were ideas for badges connecting to leaderboards and other reputation systems.
RE-APPLY + DISCUSSION: people felt there should be more that one chances to apply for a badge in case there is something missing from the submission. There was also a need to be able to discuss a submission with the person reviewing you.
META BADGE: combine a set of badges to create either a metabadge or a role within the community.
BADGE ANALYTICS: users requested the ability to see in one place analytics connected to the badges they have earned and compare that to the rest of the community.
BADGE NAVIGATION = CONFUSING: hard to move from individual badges back to challenges and specific tasks, too many clicks.
4. Wireframes for UX improvement and new features:
Wireframes for the challenge page: incorporating badges to tasks, making it so that the hook & end product, the goal, time and badges are clearly communicated.
Wireframes for the discussion page: integrate peer badges to discussion pages, peer badges are easy to award within the discussion page.
Wireframes for the task page: including bonus/surprise tasks for learners who need an extra challenge.
Wireframes for a Badge LeaderBoard
Wireframes for a Metabadge: made out of smaller badges, including an explanation from a credited member of the community of why one should own that badge. Additional sketch of a badge analytics page.
Wireframes for new challenge features (that allow dynamic assessment), such as debating - above called battling, between two teams who are competing for example for the best popcorn.js video of Capulets vs Montagues in an Interactive Narrative Challenge inspired by Romeo and Juliet.
Wireframes for new challenge features, such as creating a video tutorial as part of the challenge.
Wireframes for new challenge features, such as fixing a broken piece of code to prove that you know how to use for example “HTML5 elements”.