Eportfolios in Engineering – Advantages and Disadvantages

Recently, John Fielke talked to us about his experiences with using e-portfolios in his course ‘Computer Techniques’. This posting will highlight some of the advantages and disadvantages John found when using PebblePad with his first year engineering students.

Using PebblePad as a webfolio tool with engineering students - Advantages and Disadvantages.

Advantages:
I chose to use PebblePad as a tool for the students to use to submit their computer models and drawings. The year before, we used AssignIT where the students put all their work into one zip file and uploaded it into AssignIT, producing a series of files for assessment. By using PebblePad, I was looking at linking those files together with a bit of reflection from the students about how they completed the assigned tasks. AssignIT requires all of the student’s work to be bundled into one zip file and submitted as one file. I have used this twice now with large files and on both occasions, AssignIT has crashed as the students were submitting these very large files at the same time. PebblePad not only allows multiple smaller files to be uploaded, the students can upload these files gradually each week.

I used the profiler tool to get the students to look at their work and reflect on their learning outcomes by answering a series of questions. The profiler tool is a dynamic, real time resource so we can look at the students’ work and provide immediate feedback, and allow the students to act on this feedback prior to their work being assessed. In addition, it can actually make marking very easy because work can be assessed as it is submitted rather than in a block at the end of the course.

Students saw growth in their knowledge

Another advantage is file preservation. If the students upload their work regularly, they reduce the risks of losing files or having their files corrupted.

Students can create a personal page with a photo of themselves, a few of their aspirations and a some personal background allowing academics to get know the students on a personal level and not just their name and ID number. If webfolios are used across multiple courses, the students can just call up this page into each course webfolio.

Disadvantages:

To utilise some of PebblePad’s features the students need to upload their work regularly, but many were reluctant to do this. Few students took the opportunity to respond to the feedback we provided and make changes to their work before it was formally assessed.

Some students did not understand the importance of sending their work to the gateway. We also had the issue of students thinking they could use the one webfolio for two courses resulting in some students sending work to the wrong gateway.

PebblePad is an interesting repository for work but the way it is structured, asset tracking and asset finding is not very user friendly. Asset management issues included students deleting files and breaking links between files, or updating a file and not updating the hyperlinks resulting in missing files. All we see is what students link to their webfolio, not the work that they have uploaded into PebblePad. Additionally, some students compressed their work up in file formats not able to be opened by the university computers. The students needed to resubmit in the correct zip file format. We held a session on webfolio development but less than half the students attended that session. The students who attended the session found it much easier than those who did not.

In hindsight, I realise that I should have asked the students to submit a single zip file each week and post a reflective comment about that week’s work rather than submitting up to 10 files for a week. Accessing multiple files is very time consuming in PebblePad.

A passionate group of students absolutely disliked PebblePad as it was either too difficult to use or too simplistic in its capabilities. Others wanted to personalise it to the limits that other web authoring software allows. PebblePad did not easily allow this. The other drawback of PebblePad is that we were looking at engineering drawings and whilst it allows viewing of jpegs their resolution was not high enough for us to read these drawings very clearly and individual files had to be opened in a series of cascading menus to view their work and this became quite time consuming.

Future Directions

Some of the improvements that could be made to Pebble Pad are to give it a corporate UniSA rather than a primary school feel. I would like to see it re-badged, corporatised and the PebblePad name removed. The interfaces, the fonts and low-tech look of it do not appeal to engineering students.

I still think that e-portfolios are a good learning tool for the students and they integrate and roll everything in together. The students have a record of their learning, which they can reflect on if they want to. My advice to other academics would be to get on board and use some of these developments that the university is resourcing and not try to do these things on your own, but have the corporate body backing you. After my experience with PebblePad this year, I would love to use some form of effective, interactive, feedback providing, reflective online submission method in the future.

Eportfolios in Engineering

Today, John Fielke will be talking to us about his experiences with using e-portfolios in his course ‘Computer Techniques’, which is one of eight core courses common to all of The University of South Australia’s Bachelor of Engineering degrees.

Dr. Fielke identified three interesting areas that arose from his experience with using e-portfolios, or webfolios, for the first time in 2009 in his Computer Techniques course.

The use of online formative feedback for students using webfolios

PebblePad is a ‘real time’ webfolio tool, which allows academics to review students’ work and provide formative feedback. During my Computer Techniques course I gave my students feedback and then allowed them to update their work. However, very few students took up this opportunity. I suppose the students could be helped to see the benefits of the provision of this type of formative feedback if marks were allocated for the use of the feedback, but it is still up to the students to decide if they want to use to take advantage of this.

Using the Profiler tool in PebblePad to assist students to reflect on his/her development of the UniSA Graduate Qualities

Another part of PebblePad that I used was the profiler tool where I got the students to reflect on their learning outcomes. The profiler offered the opportunity for me to ask students questions and I tailored these to allow the students to reflect on the university’s graduate qualities. For example, I asked questions about their background knowledge of computer-aided engineering drawing before the course, and then I asked them to rate their knowledge after the course.

The profiler tool not only provides information about the students, but also allows the students to reflect on their own development. For example, I asked the students to rate the effectiveness of their learning in specific learning environments such as the computer practicals or the lectures. This allowed the students to acknowledge that they did learn differently in different modes. The best part of the profile is that you see this in real time as the students are working on it and you can use the reporting function to find out where everyone is up to at a specific time.

Students reflected on their modes of learning

The students might be inclined to put off their thinking about the graduate qualities until they near graduation, however, presenting a series of questions that they have to answer about their current personal development makes them reflect on this and helps them to think very clearly about the skills they are developing even at this early stage of their studies.

Self-assessment using the Profile tool in PebblePad
I also used the profiler tool to ask the students to reflect on their effort and learning in the course, and predict their grades. I was surprised at the number of students that self-rated themselves quite low and identified that they did not put a huge effort into the course. They included many comments to justify why this was the case. Some students did think they deserved a HD, whereas other students thought they deserved a C and suggested that they had understood most of the course but had a few difficulties here and there. Some self- rated themselves as a P2 and commented that they did not try too hard, but thought they had done enough to scrape over the line. A couple of students rated themselves as an F and explained that they had not had the time to put the effort into the course. I was very surprised at how accurately they predicted their grades, probably within one grade. It really showed to me that the students do acknowledge the amount of work they put in and they do aim for a grade. They are not all after a HD for an assignment, they might like it but in reality, they are just going to do enough work to get the grade they are aiming for.

It was very easy using the profiler tool. The reporting function allows quick and easy access to information provided by the students. However, there is so much functionality in PebblePad that we still have not used.

Wikis in Engineering

Today we have Liz Smith, from the School of Advanced Manufacturing and Mechanical Engineering, to share some of her experiences using wikis to support student project work in different courses.

We have used wikis in Engineering for the past couple of years. A couple of examples include....

In 2008 we used PBwiki with the common first year course Sustainable Engineering Practice. The students used the wiki as a place to create an online portfolio of assigned tasks. The tasks were focussed on developing the student's understanding of the engineering profession. This included collecting three on-line news articles on engineering and sustainability and commenting on each article and how it related to sustainable practice; collecting two on-line job advertisements for graduate engineering positions and summarising the attributes for which the employer was looking; developing a career plan map and discussing how the student planned to manage their ongoing professional development; reporting on the industry speakers who presented throughout the course and preparing a resume. Each of these tasks was approximately 200 word equivalents.

For more information on how we used PBwiki and what the students thought about it, check out our paper:
http://www.ojs.unisa.edu.au/index.php/atna/article/viewFile/378/288

And here are our presentation slides from the ATN Assessment conference 2008:

Liz Smith et al 2008

View more OpenOffice presentations from Diana Quinn.

In 2009 we used PBwiki for the second year course Mechanical Engineering Practice. The major assessment task for this course is the Weir Minerals Design and Build Challenge. This challenge is for nominally second year mechanical engineering students working in groups. To assist in working collaboratively, each group were required to use a wiki to simulate an engineering design file. In their wiki, students collated information related to the project, such as drawings, calculations, and meeting minutes.

It was hoped the wiki would help students work collaboratively. Here is a link to our example wiki http://meng2009-coordinator-2009.pbworks.com/

The benefits of using the wiki, as identified by the students include: the flexibility of uploading information at home, sharing and storing files in one location, displaying images and videos, and the ability to provide feedback on each other’s contribution. However, even with the wiki, the preferred channels of communication between group members were via phone and email, where email was the preferred method to share documents between group members. The benefits of using the wikis, as identified by the teaching staff include: indication of group progress, avenue for providing formative feedback and identification of group members who are not contributing.

Course Coordinators regularly checked each group’s wiki and provided formative feedback on its content. Students then used this formative feedback to further improve their wiki site. Although the formative feedback was appreciated, students’ comments suggest they would prefer regular summative assessment tasks based on their wiki to keep them on track.

Overall, I really like using wikis. I particularly like using PBwiki as you can create private wikis for free. Other wiki tools such as Wikispaces require you to pay a monthly amount to keep the wiki private.

After using PBwiki in 2008, I declared that I would never use PBwiki as my wiki tool again. But, since then they have updated the software so that it has more functionality and easier to use (it also has a new name PBworks). So when deciding on which wiki to use in 2009 I chose PBwiki – better the devil you know...

This is not an advertorial for PBwiki, it is just my wiki of choice. There are plenty of wiki tools availble, a good comparison of the different wikis available can be found here:

http://en.wikipedia.org/wiki/Comparison_of_wikis

A conversation with Peter Kentish about Moodle quizzes

I cornered Peter Kentish recently, gave him coffee, a comfy chair with a cushion and asked him to recount  some of his experiences about using Moodle quizzes to share with our readers. 

Q: How come you are using Moodle Quizzes now?

It all started in 2007 when the new common first year in engineering began. I had always taught my old Engineering Materials course with a cohort of 60-70 students using 6-7 multiple choice and text-based questions but now, with the new common first year core course it was going to be 200! There was no way I was going to be able to do all that marking! I knew  that the quiz approach had worked well for motivating my students to learn consistently in this course so I asked for an automated system to help me mark these tests. 

It wasn't the first time I had asked for an automated assessment system. The UniSAnet team tried several times but were quite frustrated in their efforts to create an in-house 

online assessment tool and resorted to trialling the Moodle version. I was one of 2 people who were allowed to use this environment and test it for the University. 

How do you use it?

I run 6 tests during the study period for this course. For each these there is actually a pair of quizzes. The first is a practice (non-assessable) one that students can attempt as many times as they like, find out which they got right or wrong and also what the right answer is. Then there is an assessable version that counts for grades.  There about 30-35 items in the assessable one and 15 or so in the practice version. 

A screen grab from the Moodle quiz environment

 

Students do the assessable quizzes in a computer pool during their tutorial time. I set the time the quiz is open for and a password - I make these up so they are different for each tutorial class. When all the classes have finished taking the test I send them their results which have been automatically marked. 

What has it been like using Moodle quizzes?

The system itself has been quite good - robust.  There have been virtually no bugs in the system - just one that hasn't been solved. This has no negative effects provided you are aware of it.

Like most things there are positives and negatives - there has been a lot of time setting up the database of questions, but the time saved with the auto marking has more than made up for it - particularly as I am now in the second year. The benefit will continue for future years as less time is required in maintaining the database.

How do you come up with your questions?

When I first started I had my old questions that needed to be entered into my database which took a long time. Now, when I want to set up a test I create a shell for it (setting the time and password for the class) and select the questions I want from my database. Each year I choose different questions. I also use the random shuffling functionality - my selected questions are presented in a different order and answer options are also in different order for each student (this shuffling feature can be turned off). 

Also, at tutorials I get ideas for questions where the content is applied to problems and write new questions to add to my database. This is good as I really want to test students application of knowledge rather than recall of memorised facts.  

Also, tutorials are not assessed directly, so to motivate students to engage with the tutorial activities I warn them that a certain amount of the quiz items are going to come from the tutorials so they need to do this work (i.e. memorisation of facts will not be enough to be successful - you are not very good as an engineer if you are not able to apply knowledge to solve problems!)

An example of a question from a practice quiz

What do the students think?

They generally like MCQ best - they think they have an advantage as the right answer is there they just need to find it.  For the first practice quiz there were over 958 attempts - some students attempted it 15 times, others not at all. The average mark for the test was about 70%. 

Even though the students were advised that quiz 2 would be more complex than the first, the number of attempts on the practice test did decrease to 795 -probably because the students have more competing interests or they have been lulled into a false sense of security from test 1. The average mark for this second test, which was more difficult,  was  lower - around 60%.