Measuring learning

I went to a seminar-y session today on evaluating training effectiveness run by Rustica Lamb from Bloom training. I found it really helpful and am very grateful that Rustica took the time to share her knowledge with us (for free!).

I had heard of, and read about, the things that Rustica discussed, such as the ADDIE (assess, design, development, implement, evaluate) model, Kirkpatrick’s evaluation measures and Blooms taxonomy but I never really got it. And it’s quite surprising (to me anyway) that I never got it before because I learn quite well by reading and absorbing information on my own.

What I think was helpful was that I was able to put it into context. I had just come from meeting someone really interesting in the Organisational Development (OD) space and we had been discussing measurements of success for OD initiatives. I am also presenting a model for analyzing the performance of HR initiatives at the HRINZ research forum next week and had been thinking about how I present this. On top of both of these things, I had been thinking about how I measure the success of my leadership development course.  

Evaluating training effectiveness was very topical for me today. Aside from that, Rustica delivered the material in a very simple, easy to follow way. I was able to listen and see where it fit into what I was thinking about.

So, what were the key things that I picked up? Firstly, I gained knowledge of how to write robust learning objectives that encompass a verb, task, condition and standard. I think I need to practice this before I can truly say I’ve learnt it.

But, I did find it alarming that I did not already know this stuff. I have a National Certificate in Adult Education (Level 5) and I never learnt about robust techniques for writing learning objectives, and therefore assessments, when I was doing my certificate.  Yet, I am ‘accredited’ to assess and moderate NZQA based material. I think I naturally write good objectives, but I do question the strength of the accreditation process in light of the lack of a structured methodology.

The second thing I picked up was the different ways of measuring the effectiveness of training using Kirkpatrick’s evaluation model. Again, I think I’ve done a lot of this naturally. In the past, I have been pleasantly surprised by outcomes such as significant business savings as a result of my training intervention. However, I do need to be more strategic with this.  In saying that, I think the measurement of business outcomes as a theory needs further development. How do you measure the degree to which the outcomes occurred as a result of the training initiative? In some cases this is straightforward. But other business issues can complicate this. 

If you’re interested in hearing about a case study that looks at a business intervention, it’s outcomes and recommendations for practitioners, register for the HRINZ research forum here (non-shameless plug). 

What are your thoughts on how to measure the effectiveness of HR/training/OD/L&D initiatives?

Tweet chat 101

A tweet chat is when a group of people all tweet about the same topic at the same time on Twitter. A hashtag is used to identify tweets in the conversation. Often tweet chats will be a regular event, for example, occurring weekly or monthly.

I'm loving tweet chats. They're such a great way to connect with like-minded and enthusiastic people from around the world. They are also a great way for participants in the #lead1201 - leadership development pilot to discuss the topic for the week.

A tweet chat can be quite intimidating at first, so here are my tips for participating in and facilitating/moderating a tweet chat.

Participating in a tweet chat:

·      Identify the moderator and follow them. The moderator is the person who will be asking questions and prompting the chat along. Pay attention to what they are asking.

·      Reply directly to moderator questions. Most tweet chats will have some structure to the questions and have 3-5 questions. You might see Q1, Q2, Q3 etc. (as in question 1, question 2 etc.) in the moderator tweets. Reply to them with A1, A2, A3 etc. (as in answer 1, answer 2, etc.). This makes it easier for other tweet chat participants to follow your conversation.

·      Include the hashtag (e.g. #Lead1201) in every tweet. This makes it much easier for tweet chat participants to follow the conversation.

·      If you are chatting with a large group, select a small number of participants to interact with. There can be several different topics going on at the same time so this will help you get the most value out of your conversation.

·      Make new tweet chat members feel welcome. If someone joins the chat who you have not seen before, say hello and tell them what we are talking about today/tonight.

·      If you are replying to a tweet, include a brief summary of what was included in the original tweet. This is so everyone knows what you are talking about. For example, “I agree that leadership self-awareness cannot be learnt”. Rather than just, “I agree”. 

·      If you like or support a tweet then re-tweet it. Similar to the previous point, a RT shows that you like the tweet but also means that other participants can still follow the conversation. Rather than “I like this”. 

·      If you want to reply to a tweet or RT, and include a comment, but there is not enough space in 140 characters. You can shorten the original tweet. Copy and paste the original tweet and delete any parts you don’t believe key. 

·      Follow up afterwards. Tweet chats are great for networking. If you had a great tweet chat with someone, follow them and send a tweet to say Thank you. You could even keep the conversation going.

Facilitating a tweet chat:

·      Advertise your tweet chat to your followers. Include the time (and time zone) and topic.

·      Make new tweet chat members feel welcome. Ask them to introduce themselves to the group.

·      Put together 3-5 questions around the topic and ask a different question every 15mins. Use Q1, Q2, Q3 etc. (as in question 1, question 2 etc.) to identify which question it is.

·      Use probing questions to ask for more details around tweets - this can also help keep discussion going.

·      If appropriate, Introduce people who are talking about similar topics by mentioning them both in the same tweet.

·      Provide a summary of the tweet chat afterwards. I use storify.com for a straight transcript.  In #tchat bloggers write up a summary of the key points - this is a cool idea.

I didn't come up with all of these tips, and I learnt ALOT from participating in other tweet chats. So I'd like to give credit to Carolyn Hyams (@chyams), Firebrand - for tips on using twitter here. Also, Meghan M. Biro (@meghanmbiro), Talent culture - for modeling an awesome tweet chat conversation on #tchat (Thursdays at 12noon, NZ time).

If you are interested in participating in a tweet chat you can join #lead1201 on Tuesdays at 7pm (NZT), or #tchat is another great one. I'd also like to set up a tweet chat for NZers talking about social media for learning at #nzlearn. Let me know if you are interested in participating in this.

I might do a Twitter for learning 101 blog next week so stay tuned.

My leadership values

This week participants of the leadership development pilot are asked to write a blog about their values. What are they? Why are they important? And what they’ve done when their values have been challenged?

This is a personal reflection exercise and I’m not ignorant to concerns about portraying oneself publicly. This week I’d like to lead by example and share my values on my blog, including where they’ve been challenged.  My main values are:

Integrity

Wikipedia defines integrity as  “the honesty and truthfulness or accuracy of one's actions”. To me, integrity includes honesty but only when that honesty doesn’t conflict with my overall integrity. For example, if someone confides in me and asks that I keep it to myself I do just that, even when asked point blank.

Integrity also means standing up for what you believe in. If I don’t agree with something, or someone, I generally express my views. If I don’t it’s because I’m either being polite, shy or avoiding conflict. When I do avoid conflict, I see this as a weakness.

Transparency

I’d like to think that what you see is what you get with me.  For a while I did work at being less transparent. I still believe I need to be a bit more diplomatic about how and when I say things, but that doesn’t mean I shouldn’t express it.

Integrity and transparency go hand in hand for me. I have left a job because I felt they were being threatened. I felt I was expected to engage in personal gossip about employees that colored manager perceptions and influenced decisions. I actually stayed in the job a lot longer than I should have but it caused me much stress and anxiety.  

My leadership Brand

My values represent my leadership brand. These are the things that I want to be known for. I am trying to make my brand quite explicit but I am also opening myself up to criticism.

I don’t believe I embody those values wholeheartedly. There are many things I need to work on. I also need to achieve a balance between them. I hope to be transparent (there is that word again) about my weaknesses.

My values define the behavior and ethical boundaries that shape me as a leader. They also contribute to the achievement of my leadership purpose. They are the wheels on my car that help me go in the direction I want to go.

So what are your values? Why are they important? And, how do they shape you as a leader?

Facilitating social learning

We’re now in week three of the social media leadership course. There has been a lot of enthusiasm for the course from participants that has been absolutely wonderful. However, I am now a pattern that is not uncommon with traditional forms of training.

You get the star performers who actively participate and don’t require any pushing or follow up, those that are lurking in the background and will get what they need to get done done but at the last possible minute and those that you don’t hear anything from and, despite follow up, you don’t really know how they are doing.

It is the last two categories that are the hardest to engage. Particularly when you’re not face to face with these people. One of the ways I’ve worked with this in the past is to encourage the managers of training participants to engage with their people around their learning. This requires taking on a coaching role – what is going well? What is not going well? What are you learning? This is a potential option if I were to do the course again, but maybe they nominate a coach who engages with them through social media? Oooh this could be someone who has done the course already, thus giving the ‘coach’ ongoing development as well (thinking out loud).

The biggest challenge in engaging learners through social media is the facilitation of the course. The point is not to provide the participants with content by way of information about the topic, but to encourage them to seek it out for themselves. Some are very adept at this, some not so much. Some of my facilitation has worked well, some not so much. Although it is very tempting to provide the information, I’ve scaled back the amount of information I provide. Several participants have stepped up to the plate to feed information to the twitter stream and this has been fantastic. I have also started doing weekly tweet chats, to provide a time and place for the sharing of ideas – this seems to be working really well. Tweet chats are Tuesdays at 7pm, if you’d like to join us.

However, there is one thing in particular that I would do differently right from the start of the course and that is to provide a lot more information about how blogs, Twitter and LinkedIn works as learning tools. I think I’ve picked up the learning potential of them quite intuitively, as I think a couple of participants in my course have. But it is not really obvious to everyone unless you’re that way inclined. So I’m going to put together some guides on how to use social media for learning.

There are some really great resources out there already on how to use Twitter (see Kirsti Grant from Social Sauce) and how to write blogs (see Tanja Gardner from Crystal Clarity Copywriting) but not much from the perspective of “how do I learn from social media?”

Watch this space J