I originally wrote the following blog post in January 2014. I am re-publishing it here given recent events and discussions I have had on the views of employers in relation to Open Badges.
Doug Belshaw also wrote a post last week which highlights that some of the challenges discussed in this post, remain perceived challenges today. His post, ‘The three biggest (perceived) problems with Open Badges‘, challenges the thinking behind these perceptions.
There was an exciting announcement earlier this week by the IMS Global Learning Consortium on their commitment to accelerating the adoption of Open Badges in formal and corporate education: IMS Global Announces Initiative to Establish Digital Badges as Common Currency for K-20 and Corporate Education
Doug has also written a useful post on this, ‘An exciting week for Open Badges‘, which unpicks some of the reasons why this is an important announcement for the uptake of Open Badges. It also provides further evidence as to why expanding our thinking in relation to digital credentialing and Open Badges is becoming increasingly important.
One of the ideas mentioned in the post, around clusters of badges, was also explored further during the Discovery project, a proof-of-concept project on badge based pathways to employment that I worked on with Mozilla last year: Discovering pathways
Concerns, Possible Solutions, Paradigm Shifts and Key Findings
There was focused and lively discussion at the Open Badges: Ways to Evidence Employability event I co-facilitated at the University of the West of Scotland (UWS) this week.
Alongside Tom Caira, Chair of the Industry Advisory Board for Computing and Frances Rowan, Stakeholder Manager (Careers and Employability Service) from UWS, we brought together over 50 technology employers (Directors from multinational and local companies), students, educators and members of the
Open Badges in Scottish Education Group (OBSEG) to consider if Open Badges could provide a useful way of highlighting employability skills and attributes employers are looking for.
Commencing with an ice-breaker activity we split into groups and asked students to say what attributes they thought employers might look for in potential employees, then employers commented on what they are looking for. This was followed by presentations on the key attributes research is showing employers are currently interested in and a collated list from the employers at the event.
 |
List of employability skills
Attitude; Communication; Business Awareness; Resilience; Reliability; Problem Solving / Dynamic Thinking; People Skills; Responsibility (team working and self-awareness) |
I then gave an introduction to Open Badges and we followed this with the groups using
DigitalMe’s Canvas Design Sheet for the task of considering how they could evidence these attributes using Open Badges. All of this sparked a barrage of questions!
Concerns
As you perhaps might expect from this group of people, the questions were very pragmatic and focused on the practicalities of using Open Badges in the employment process. During the resulting discussion people voiced their concerns and identified what they would need in order to find value and place trust in badges.
An initial concern was investing in something they thought might create an additional administrative burden. There was agreement that they would not wish to be presented with so many badges that it would result in additional time or resources required for the application process. If in the process of reviewing badges, they also came across badges for things that they didn’t feel were of value or relevant to the application, they felt this could undermine the value they placed on all badges.
Some employers were also concerned about the open nature of the infrastructure, with one employer saying it scared him. Another commented he would want to see people licensed to issue badges. The underlying concern here was that employers don’t have time to deal with a hit and miss scenario with badges. Currently when they see that someone has a degree or certificate from a known educational provider, they trust the value of this because they have an implicit understanding of the quality assurance processes and verification procedures the awards from these institutions have gone through. They will also value certificates from some institutions or training providers more than others, again based on their own trust networks, which will have been built up from their own experiences, the views of their network, the sector and generally accepted social views of top educational providers. Badges add a new dimension to this mix because they could come from issuers the employers have not heard of so they would have nothing to base immediate judgements on as to whether or not it would be worth clicking through to find out more about the badge and why it was awarded. They also weren’t sure how recruitment agencies would deal with this. The last thing the employers wanted was to add to the length of time it takes to get through a sometimes very large pile of applications for a job.
There were questions around the potential for inconsistent use of words to describe what is being badged. For example, there could be inconsistencies in people’s perceptions of what makes up an attribute. One of the attributes we were working with was ‘resilience’, which could be interpreted in different ways and result in badges being issued based on divergent understandings of what resilience means. Badges could also be issued to someone for showing high levels of resilience but if that word didn’t appear in the title of the badge, how would this be picked up until the badge consumer drilled down into the criteria and evidence? The employers were clear that unless the underpinning reason for the badge was obvious, they might not bother to click through and read this additional information.
Finally, as participants began to work through the badge canvas, they began to appreciate the challenge involved in evidencing an attribute. How do you demonstrate good listening skills, resilience or commitment in a way that can be consistently verified? How can you capture something that becomes evident through the actions someone takes over a period of time or that they might demonstrate by their attitude and approach to a variety of situations?
I think the key concerns from this group of employers could be summed up as:
- Employers want to immediately understand the value behind a badge. They don’t want to spend time clicking through to the detail of a badge unless they feel it will reveal something worthwhile
- Employers are concerned with badge apathy and are likely to be put off the concept of badges if they come across too many badges that are irrelevant to them in a given context
- It is probable that new trust networks will develop but initially it is likely badges from issuers employers already know and trust will be valued more.
Possible solutions
There are implications for issuers developing a badge brand and building up the value in their badges. Once employers begin to recognise the look of a badge from a particular issuer, and if they have valued other badges from that issuer, it seems more likely they will click through to the information behind another badge from that issuer. During discussion with individuals after the event, ideas like having a way to endorse a badge to say the badge consumer found it useful, relevant etc were raised. Rating systems can pose problems, such as people indiscriminately giving good or poor marks or reviews but perhaps it is worth exploring. The concept of endorsement did cause some confusion during the event when I raised the potential for others to endorse a badge, such as a company endorsing a badge issued by an educational institution and vice versa. Some thought this meant anyone could endorse a badge, which as mentioned above, might result in inaccurate endorsements. Once I had clarified that the endorsement would take place before the badge was issued and would essentially demonstrate that an external party was recognising and endorsing the value of the badge and the rigour of the processes applied in developing and issuing it, this was seen as being of more value.
 |
| From Flickr by Doug Belshaw CC-BY-2.0 |
For badge display developers, we discussed that something which would potentially help employers / recruiters to make judgements about attribute strengths is cluster displays of badges eg like the honeycomb image you often see associated with badge related posts. Clusters of badges could be generated through badges being tagged with keywords. So if a job applicant has a number of badges tagged with the term resilience, these would display in a ‘resilience’ cluster, which could provide a quick visual clue for recruiters to explore that candidate’s application further. This resulted in discussion around whether or not a taxonomy of attributes would be helpful so that there is a common understanding of what being resilient means. This cluster approach could also perhaps give some level of security that to have received so many resilience badges, this candidate does have some strength in that area, even if none of the badges come from issuers in the recruiter’s existing trust networks. This kind of development, could allow employers to extend their trust networks by being introduced to new issuers that they have already given some, albeit potentially minimal, level of credence to.
With respect to evidencing attributes, a number of the students at the event were mature students, some of whom were juggling young families, a job as well as a Masters degree. Someone managing their studies in amongst these other demands, is likely to be demonstrating quite high levels of commitment and discipline so how could this be badged? Could a possible solution be to pitch for a badge for a particular attribute? This would in a way, mimic providing information at interview where the candidate might talk through how they feel they have demonstrated a particular attribute by how they have approached different situations, their overall attitude etc. If this was presented for a badge, the pitch could be verified by someone others would trust to make that kind of judgement. (The information as to why they are in a position to make this judgement could be included in the badge criteria, eg that they are a course tutor or community leader who has known the person for a period of time.)
In terms of avoiding badge apathy or de-valuing badges, the implications for badge earners applying for jobs would be to pick and choose the most pertinent badges to present for a particular job application.
Paradigm shift
Beyond the practical questions from participants, the concept of Open Badges sparked some passionate debate as people grappled with the potential impact badges could have on our existing system of assessment and award. On one level, Open Badges can simply provide a digital equivalent to paper based certificates to show that someone has achieved something. On another level, they could also provide an opportunity for a paradigm shift in how we assess, and what we recognise, in terms of an individual’s skills and attributes.
The digital nature of Open Badges creates a variety of opportunities that don’t exit, at least to the same extent, with paper-based assessment and certification. Currently, candidates can make all kinds of statements about their attributes in a covering letter, that a recruiter, if they do not know the candidate, generally has to take on trust. There may be no immediate way to ascertain whether the person is actually a good team player, problem identifier and solver etc until perhaps the interview stage or a phone call to someone who knows the candidate. With Open Badges, the ability to link to digital evidence means the recruiter can, with a click, immediately see and make their own judgements about whether or not the earner has deserved a badge issued for a particular attribute. Links to evidence or the fact that the candidate has received a badge from an issuer the consumer trusts, could help to narrow the field of people to invite to interview by giving some more weight to the statements made by the applicant.
There are also opportunities around ‘big data’, which could be used to surface things like how many learning scenarios someone has undertaken on a particular subject, how many online quizzes or other forms of assessment they have successfully completed, their contribution to a community’s knowledge etc. Why not issue badges based on what is surfaced by some of this data? Stephen Downes has argued in a post on how we assess, that perhaps we should look at assessing individuals for their contribution to a community’s knowledge. This contribution could be identified through blog posts, tweets and comments etc that someone is doing anyway, without them thinking about them being used to demonstrate aptitude within the community. I personally like the idea of assessment becoming more invisible – something that can take place without people really being aware of it (Helen Keegan’s exemplary stealth assessment scenario at the University of Salford, I think demonstrates the potential power of this approach well).
Job seeking and recruitment through the online environment is increasing and online professional profiles can work 24/7 to promote an individual to potential employers and recruitment agencies. By adding Open Badges that highlight attributes alongside formal qualifications on these profiles, recruiters could gain a more holistic view of an individual and search for those individuals with just the right mix of skills and attributes they seek.
Key take-aways from the Evidencing Employability Skills with Open Badges event
The employers who attended the event are interested in the opportunities Open Badges could provide but with conditions around value and trust. Value and trust could be built by consideration of the following:
Relevancy
The employers want to see badges which are meaningful and relevant to the context, eg to only be presented with badges relevant to the skills and attributes for a particular job being applied for. They believe badge apathy is likely to develop if they come across too many badges they feel are irrelevant in a given context.
Rigour
In order to trust the value of a badge, the employers want to feel secure in the knowledge that appropriate rigour has been applied in developing, assessing and issuing a badge. For example, they want to know they can trust the validity of the processes the issuer has gone through in developing the badge, that the appropriate weight or level of effort / knowledge / aptitude is indicated and that adequate quality assurance processes have been adhered to while developing the badge.
Consistency
Connected to rigour, they want to know that the assessments or evidence required for achieving a badge have been consistently applied and the same badge is issued for comparable levels of effort, achievement or ability. They are also interested in visual consistencies, so that they can recognise badge brands to build up quick visual references of trusted issuers and also understand different levels of achievement via the badge image.
I think the key points for issuers and badge earners is that the trust and value employers place in Open Badges, could be built up over time through rigorous and consistent processes being applied when creating and issuing badges and through the earner being selective when choosing relevant badges to present for a given context. In essence, I don’t think this is very different from how trust has been created in the current issuers of awards or how we re-write our CV / resumes for different contexts.
Tom Caira concluded the event by asking if people would endorse them introducing Open Badges as a pilot to an entrepreneur module. This resulted in a cautious yes from the participants as they continued to reflect on what they had just learnt during the workshop and someone suggested a Return on Investment exercise to try to gauge the effort required in creating the badges compared to the possible gains in terms of helping students demonstrate employability attributes in a way that employers value. On the whole, however, there was consensus that it would be useful to explore Open Badges further, run a pilot and have some feed in from employers during the process.
Grainne's blog 