Data-Based Decisions: Master of Technical Communication

2020-21   2019-20   2018-19   2017-18   

2020-21

Data-Driven Decisions

As we noted in our last assessment, assignment-based assessment continues to presented a challenge. The first learning objective (L1: Demonstrate practical technical communication skills) presents some disagreement of what deliverables the assessment committee expects to evaluate. The core issue is how instructors meet L1 depends on the class. For example, outcomes could vary depending on the course (editing outcomes for an editing class, research report outcomes for a research class, etc.), making it difficult to assess student progress on the goals.

Tweak the Assessment Tool. It is clear that the assessment tool needs adjustment. While we were unable to tweak our assignments this year, going forward, we will institute the following assessment changes.

  • ENGL 6400: Editing will be evaluated on L1, L2, and L3
  • ENGL 6410: Introduction to Technical Communication will be evaluated only on L1
  • ENGL 6830 Introduction to Rhetorical Theory will only be evaluated on L2

We have also revised the tool itself to clarify course goals.  

  • L2 becomes L1 (Understand)
  • L3 becomes L2 (Evaluate)
  • L1 becomes L3 (Apply)

Instructors are encouraged to design the assignment prompt used for assessment with these three literacies in mind.

Students Request Varied Content. At previous assessments, we noted that students asked for differing formats for discussion. Rebecca Walton introduced Flipgrid videos post and responses to put into the mix. Students have also asked for different media and genres. For some classes it is not appropriate, but some courses have mixed zines, videos, collaboration exercises, to name a few. 

Evaluate Content Delivery. Given the current pandemic conditions, with distance working and learning widespread, we continue to consider piloting synchronous learning in MTC. Though the program has been asynchronous, with the increasing ubiquity of Zoom instruction, we are exploring ways to bring in synchronous learning, while still considering the original intent of the program where time zones and work situations make synchronous learning impossible. Going forward we will continue to look for ways to integrate synchronous Zoom into our classes.

2019-20

Data Driven Decisions

Assignment-based assessment once again presented a challenge. The first learning objective (L1: Demonstrate practical technical communication skills) presents some disagreement of what deliverables the assessment committee expects to evaluate. The core issue is how instructors meet L1 depends on the class. For example, outcomes could vary depending on the course (editing outcomes for an editing class, research report outcomes for a research class, etc.), making it difficult to assess student progress on the goals.  

Tweak the Assessment Tool. It is clear that the assessment tool needs adjustment. In next year’s assessment, we will institute the following assessment changes.  

  • ENGL 6400: Editing will be evaluated on L1, L2, and L3
  • ENGL 6410: Introduction to Technical Communication will be evaluated only on L1
  • ENGL 6830 Introduction to Rhetorical Theory will only be evaluated on L2

We have also revised the tool itself to clarify course goals.

  • L2 becomes L1 (Understand)
  • L3 becomes L2 (Evaluate)
  • L1 becomes L3 (Apply)

Instructors are encouraged to design the assignment prompt used for assessment with these three literacies in mind.

Students Request Varied Content. At last assessment, we noted that students asked for differing formats for discussion. Rebecca Walton introduced Flipgrid videos post and responses to put into the mix. Students have also asked for different media and genres. For some classes it is not appropriate, but some courses have mixed in zines, podcasts, and videos.

Evaluate Content Delivery. Given the current pandemic conditions, with distance working and learning widespread, we are also considering piloting synchronous learning in MTC. Though the a program has been asynchronous, with the increasing ubiquity of Zoom instruction, we are exploring ways to bring in synchronous learning, while still considering the original intent of the program where time zones and work situations make synchronous learning impossible. Going forward we will continue to look for ways to integrate synchronous Zoom into our classes.

2018-19

Data Driven Decisions

Based on the measures above, the Technical Communication and Rhetoric faculty worked with department leadership to offer a summer graduate course in 2019 as a “test” course to gauge demand. 14 students registered in total, 11 of them MTC students. This suggests that there is significant demand for summer courses and that MTC students would benefit from regular, though limited, summer course offerings: e.g., 1 summer course every other year. Faculty have also revised their assignments, particularly in the ENGL 6410 introductory course, to provide opportunities for students to develop materials that may be useful in their work or other communities of which they are a part. Two of the three major assignments in ENGL 6410 were redesigned entirely to support students in making connections between the scholarship of the field and communities (such as workplaces, professional organizations, and others) of which they are a part. Fall 2019 is the first semester these assignments are being piloted, and we will evaluate their effectiveness in meeting MTC student needs next year based on the quality of work produced as well as student feedback.

2017-18

Based on the direct and indirect measures, the Technical Communication and Rhetoric faculty have worked with department leadership to be able to offer one summer graduate course in 2019 as a “test” course to gauge demand. In addition, faculty have begun revising their MTC assignment descriptions to build in a range of options that address the same learning objectives while mapping more closely to the work environments of industry practitioners.

Notes on Evaluation of Problem A (Nov. 2017): We have monitored the students taking ITLS courses. While the ITLS syllabi sufficiently address L1, students have expressed concerns with the rigor and lack of community in some of the ITLS courses.

A. Data indicating an issue: As described in the Program of Study assessment of individual students, all 13 students graduating from Fall 2014 to present have taken a number of classes with Dr. David Hailey, who has just retired. Dr. Hailey offered several classes that gave high priority to learning objective one, “practical technical communication skills,” including ENGL 6420, 6430, 6450, 6460, and 6470. With Hailey’s retirement, there was concern that students would not meet their skills learning objective (L1) satisfactorily.

Discussion: Anticipating this problem, the Technical Communication Curriculum Committee met (2/3/16; 3/2/16; 4/13/16) to discuss how, when Dr. Hailey retired, students would lose a number of course offerings with high priority given to learning objective one, which focuses on practical technical communication skills. Specifically, Dr. Hailey often taught 6430, 6460, and 6470.

Solution: The committee decided to pursue a partnership with the ITLS (Instruction Technology and Learning Sciences) Department, which offers many practical skills courses online. Faculty discussed the possibility with the head of the ITLS program, who agreed that they would allow MTC students to take certain ITLS courses in 2016-2017, and then they would meet to assess the impact on their ITLS program. Students from the ITLS program would also be welcome to take courses in Technical Communication. Beginning in Summer 2017, students can take Internet Development, Digital Video Capture I and II, Graphic Production I and II, and Grant Writing, all of which address MTC’s learning objective one.

Evaluation of solution: In our April 2017 Technical Communication Curricular Committee we will do the following:

  • Count the number of MTC students enrolled in ITLS courses (Fall 2016; Spring 2017)
  • Evaluate the syllabi from the ITLS courses to ensure that the ITLS courses sufficiently address L1
  • Survey MTC students taking ITLS courses regarding their experience of L1

B. Data indicating an issue: As shown in the table in Outcomes Data (add link here) students have been disproportionately registering for 6400-level courses over 6800-level courses. This imbalance suggests that students may not have been developing sufficiently broad foundations in rhetoric, composition, and pedagogy (fulfilled in 6800-level courses).

Discussion: Recognizing the importance of breadth in the MTC Program for achieving L2 and L3, members of the Technical Communication Curriculum Committee met on April 13, 2016. The following day the chair of the MTC program met with the Director of Graduate Studies to share our potential solution to the concerning pattern revealed by the data.

Solution: The committee proposed to make 6830 (Rhetorical Theory) a required course for all MTC students beginning the program in Fall 2016 or subsequent semesters. 6830, which historically had sometimes been offered as a face-to-face-only class, will now always be offered online to ensure accessibility to all MTC students.

Evaluation of solution: All new students entering the program from Fall 2016 forwards will have their programs of study checked for inclusion of 6830.

C. Data indicating an issue: As shown in the table in Outcomes Data, some students have been repeating for credit two courses (6460, 6470), which have been predominantly taught by the same two professors, to fulfill the bulk of their degree requirements.

Discussion: This repetition of particular instructors and particular courses could reflect an overemphasis on certain approaches to fulfilling L1. Recognizing the importance of developing a variety of practical skills and exposing them to a variety of pedagogical approaches, the Technical Communication Curriculum Committee met on March 2, 2016.

Solution: In the March 2, 2016 meeting, the Technical Communication Curriculum Committee decided to develop a rotation of courses and instructors that would ensure that MTC students would be exposed to a greater variety of pedagogical approaches. This rotation will be implemented in November 2016 when the committee meets to plan course rotation for the 2017-2018 academic year.

Evaluation of solution: When the Technical Communication Curriculum Committee meets in November 2017 to plan course rotation for the 2018-2019 academic year, we will also review the actual course offerings in the previous academic year to make sure that they did in fact follow the planned rotation (check for canceled classes, etc.).

D. Data indicating the problem: According to our evaluations of final projects in the core MTC courses, some students scored low in demonstrating L3 (ability to critically evaluate workplace practices and suggest new practices in light of theory and scholarship).

Discussion: While a greater percentage of students scored high on the L3 objective than in the mid- or low-range, our goal is to have all students score at least in the mid-range. After discussing the objectives of final projects, however, we found that an engagement with theory was not as specific in the requirements of some classes (e.g., Advanced Editing) as it was in others. This may be the reason for the two low scores in the L3 category.

Proposed solution: In the November 1, 2017 Technical Communication Curriculum meeting, we decided to add a requirement to each of the core courses that asks the students specifically to justify their workplace practices and suggest new practices in light of theory and scholarship.

Future Evaluation of solution: We will evaluate the solution in our 2017-2018 assessment to see if fewer students score in the low-range of the L3 objective.

E. Data indicating the problem: In individual conferences with students we have solicited feedback about the Master of Technical Communication program at large. Feedback from students about their experiences in the program is generally positive. When comparing MTC courses to electives outside the program, students have said that they like our classes better, particularly the community of learning aspect. They also like that we give them some freedom to work on projects related to their own interests in many of our courses. Nevertheless, students have expressed concern that we do not offer enough courses each semester for them to graduate at the pace they desire. Also, some have desired to take summer courses, which we do not currently offer.

Discussion: In the November 1, 2017 Technical Communication Curriculum meeting, we discussed potential options for addressing this problem. The main challenges to solving this problem is the amount of MTC faculty we have to teach extra courses and making sure summer or extra classes will fill. We do not have huge waiting lists on the classes we currently teach. Also, this could be a problem that is just a temporary anomaly.

Proposed Solution: In a November 1, 2017 meeting, the Technical Communication Curriculum Committee decided not to take direct action yet, but to monitor the situation again in the next year by soliciting feedback and closely watching registration numbers in order to determine that the desire for new courses is not just a temporary anomaly.

Future Evaluation of the solution:  We will again solicit feedback from students about the program and closely watch registration numbers. Should these measures continue to show evidence that we need to offer more courses, we will work to make extra courses available.