The Problem

Due to the COVID-19 pandemic, NYU Langone Health’s leadership decided that all non-patient facing workforce will be working remotely on a rotation basis. Qualifying staff, organized in 4 groups by workstation location, are expected to be present in the office 25% of the time, while the other groups work remotely. The Organization Development and Learning Department, as well as other institutional education departments (such as Research Education, Nursing Education, Finance and Epic) must conduct their meetings and trainings via Cisco WebEx.

As per the guidelines and recommendations of the Information Technology Department, Cisco WebEx^® Training is the designated tool for conducting training via videoconference. WebEx^® Training is Cisco's solution to virtual educational settings (as opposed to Cisco WebEx^® Meetings, which only provides basic features for conducting meetings). Managers, training professionals, instructors and leadership staff must use Cisco WebEx^® Training to for all the educational events that involve human interaction.

Due to the abrupt nature of the emergency transition from in-person training to virtual, many staff members struggle with selecting and properly using the right tools for training due to having to make a sudden change without any formal training. Additionally, since WebEx^® Training was deployed after the effects of the pandemic, the vast majority of staff in charge of delivering training and regulatory education are unaware of and/or uneducated on WebEx^® Training as a tool to deliver quality virtual training. They are also unaware of the difference between WebEx^® Meeting and WebEx^® Training. Furthermore, Cisco’s job aids and online tutorials are neither adequate nor up-to-date enough to have staff rely on them for learning how to use their applications.

Another Problem

Learners – Needs Assessment

Guiding Questions

• What activities do you conduct during in-person training that you might want to learn to do using WebEx^® Training?


• How comfortable are you using technology to deliver training virtually?


• What support do you need to setup and deliver training using technology?


• Have you conducted training using videoconference tools in the past? If yes, what challenges did you face?


• Do you use interactive exercises when conducting training?


• How do you plan to use WebEx^® Training?


• Are there any specific WebEx settings that you might need support (i.e., sharing local computer audio when showing a video)?


• Do you know how and where to find technical support if/when needed?

Learner Gaps

The Solution and the Learning Environment

Solution Logic

The Solution in Context

As part of the efforts to protect their non-patient facing staff from the effects of the pandemic, NYULH designated financial resources to provide equipment, specialized devices and software for remote work, for all non-patient facing employee (an estimated 12,000 employees).

The WebEx^® Training workshop is one of the components of this effort to support staff working remotely. Employees received guides and one-on-one support from the IT Department, to setup and install the required hard/software for communication with the institution (I happen to be the designer of the equipment setup guide).

Learning Environment

The agile instructional design approach for WebEx^® Training provides the means for the following regarding the learning environment:

• Learners will receive hands on training via WebEx^® Training, from their remote workstations.


• We expect that learners will learn by doing and participating in WebEx^® Training as training attendants.


• The Facilitator will integrate any scaffolding received as part of the learners’ takeaways for instances when such scaffolding is needed, to build a knowledge base and create job aids for future use.

Learning Goals

• Learners will be able to use WebEx^® Training tools to schedule training, invite trainees, and manage training events integrating WebEx^® Training and the Learning Management System.


• Learners will be able to seamlessly conduct training using Cisco WebEx^® Training by selecting and using the appropriate features as needed (role assignment, breakout rooms and interactive tools).

The Learners

Learning about virtual teaching in a virtual learning environment

The vast majority of the this workshop learners are known colleagues who already have established relationships with our training department. The overall makeup of the target audience is listed in the table below.

Taking into consideration the above-mentioned audience factors, the workshop contains a Q&A session, along with job aids that can be used for future reference.

Learners' Expectations

• Learners plan to use the different features demonstrated in the workshop in future sessions that they will conduct.


• Learners want to know how to use the available technology during the different phases of training, that is, planning, organizing, delivering, evaluating and reporting.


• Learners expect that this technology will enhance processes involving tracing attendance and sharing resources.


• Learners wish to maintain the same level of quality instruction that they have during in-person training while using the new virtual format.

The Objectives

• Distinguish the different Cisco WebEx^® tools for meetings and training


• Schedule training events using WebEx^® and FOCUS (Learning Management System)


• Prepare a training session:
• Assign roles
• Grant permissions
• Share Files


• Select and use WebEx^® Training tools to conduct training:
• Polls
• Breakout Sessions
• The Whiteboard

The Logic Model

Working backwards, starting from ensuring that NYU Langone Health complies with the required regulatory education for their employees as it also maintains its reputation of a “World-class, Patient-centered, Integrated, Academic Medical Institution”, I started with establishing the desired long-term, intermediate term and short-term outcomes.

The WebEx^® Training workshop is one of the components of a larger, collective effort to support staff working remotely. Aside from having received guides and one-on-one support from the IT Department to setup and install the required hard/software for communication with the institution, employees are to receive support and training on how to conduct educational events using WebEx^® Training. The Input section of the Logic Model contains all the necessary resources and stakeholders required for the WebEx^® Training component of the initiative.

The combination of the above-mentioned resources, the agreements made with Cisco (the telecommunication company of choice), all the parties involved in working remotely, and all the institutional entities that must provide ongoing regulatory education became the Input items in the logic model.

The necessary activities listed pertain only and exclusively to learning the use of WebEx^® Training. That is enforced by having established the requirement of having the necessary remote work equipment installed and functioning before taking the workshop. Employees cannot take this workshop without proof that they have the appropriate equipment, devices and software setup in their remote locations.

The Learning Analytics and the HR Analytics Departments contribute the data that reflects the outputs. As more employees and departments increase their awareness, knowledge and skills of delivering educational events using WebEx training, the data should show an increase in: (a) attendees of the workshops, (b) usage of WebEx training, and (c) number of WebEx training sessions scheduled.

Analysis

Learning Gaps

The training addresses only the gaps that are highlighted in the table below. The reasons for not addressing the other gaps are:

• They are beyond the scope of the goals and objectives.


• They are addressed in other components of the larger initiative.


• They will be addressed when measuring impact.

Analysis

Learning Goals

Analysis

Instructional Design Method 1: SAM

Since April 2020, when the training started, several iterations of the training content have been implemented, each one enhancing the content, details on particular features as requested, addition of job aids, and updates in the interface. The savvy start is where we gathered all the necessary information from Cisco, the IT Department and the learners. As the training sessions progress, there are brainstorming sessions where the design is revised, tested and implemented. Those actions align with the SAM Model, where the design iteratively improves until the target measures of performance and success are reached.

Analysis

Instructional Design Method 2: Backward Design

Another way of looking at models used to develop this training is using the Backward Design model. Our work started by already knowing what the end result should be. WebEx^® Training is expected to replace in-person training (at least for the duration of the current work situation). That means that the tasks that learners should be able to do with after the training are the same or similar to the ones that they were accomplishing during in-person training. Though there wasn’t any rubrics or formal assessment, the training functions as a community of practice, where skills and knowledge are shared, and where the more experienced learners scaffold others as they learn by doing. The required support materials and demonstrations were then created.

Analysis

Learning Theories

• Cognitive Load Theory (Sweller, 1988) - This theory is relevant since learners are provided with step by step walkthroughs (worked examples) throughout various points of the training after introducing the different features of the software. (e.g. learners are given step by step instructions on how to access the whiteboard) to minimize unnecessary cognitive load on learners.


• Chunking (Druckman & Bjork, 1994) - This theory is relevant since presenting how different tasks are accomplished by the different features of WebEx Training is extremely important for learners to understand the relevance and use of each feature. Each feature requires attention separately, and therefore, each feature is connected to a given task that can be accomplished during a training session.


• Situated cognition/cognitive apprenticeship - This theory is relevant since the instructor imparts her expertise on learners by providing demonstrations, followed by a guided exercise and a discussion or reflection on the topic.


• Scaffolding - This theory is relevant since the instructor will provide support to learners when they explore and use the new learned features of the software, so they grow within their ZPD. An additional form of scaffolding is provided in the form of job aids, online tutorials and support sessions conducted as needed.


• Cognitive Theory of Multimedia Learning (Mayer, R. E., & Fiorella, L., 2014) - This theory is relevant since both the final learning and the training itself is about using multimedia resources for delivering training.

Analysis

Learning Environment

Learners are receiving training via WebEx^® Training and from the remote workstations that they will be using for conducting training. This setup provides the advantage of having learners learn by doing. This creates a socio-constructivist environment that promotes learners developing their own relationship with the collective knowledge by observing others participate, and by contributing their own scaffolding during the training. It is expected that later, they transfer the learned skills into their own trainings.

Collective knowledge building becomes evident by the scaffolding provided and received throughout the course of the workshop. The collective actions of active participation, and the scaffolding provided by the Facilitator and other attendees help all attendants work together in accomplishing the larger task of running a successful training. As a result, employees learn skills in a context that replicates situations in which they will need to deliver training competently. This too goes in alignment with socio-constructivism, where “a critical element in fostering learning is having students carry out tasks and solve problems in an environment that reflects the nature of such tasks in the world.” Consequently, the tasks, the environment, the objectives, and the resources, all align perfectly with the instructional goals.

Assumptions

The two most important assumptions made by the Instructional Designer are based on the learning theory of constructivism and the assumptions of andragogy.

Assumption 1: Learners integrate their prior knowledge with what is learned in this training. In alignment with constructivism, where “students take what they are being taught and add it to their previous knowledge and experiences, creating a unique reality that is just for them”, it is expected that learners of this training already possess knowledge of how to conduct a training session, how to communicate via WebEx, and how to use virtual communication equipment.

Assumption 2: All six assumptions related to adult learner motivation posed by andragogy are made in this training, where “adult learning is based upon comprehension, organization and synthesis of knowledge rather than rote memory”*. In alignment with those assumptions, is expected that:

• Learners need to know the reason for learning something.


• Experience (including error) provides the basis for learning activities.


• Learners need to be responsible for their decisions on education; involvement in the planning and evaluation of their instruction.


• Learners are most interested in learning subjects having immediate relevance to their work and/or personal lives.


• Learning is problem-centered rather than content-oriented.


• Learners respond better to internal versus external motivators.

Implementation Analysis – Live Workshops

The idea of a community of practice/apprenticeship approach to this training is the perfect approach for this particular project, considering the sense of urgency and the abruptness in which this project came about. Similar to the choice of an Agile design model, the choice of media delivery is the correct one for this situation. Using WebEx^® Training to teach WebEx^® Training promotes the learning-by-doing approach, important in this type of learning. Using a different way of teaching this software tool (i.e., using a different video conferencing software to deliver the training) might have added to the learners cognitive load, as they have to use one system to learn another.

The delivery of the course via WebEx^® Training also aligns with Dual Code Theory, which establishes that both visual and verbal information is used to represent information. During the delivery of the training, the instructor makes demonstrations and provides visual representations as she verbally draws learners’ attention to the particular part of the interface she’s explaining. A good example of that is the portion of the training shown here.

Implementation Analysis - Job Aids and Performance Support

In addition to the workshop, we created job aids and provided additional performance support via a workspace in the internal organization employee portal.

The job aids comply with the following Principles of Multimedia Design:

• Coherence - Extraneous material is removed, making learning more deeply accomplished.


• Signaling - It highlights essential material to draw learners’ attention to the focus of emphasis.


• Spatial Contiguity - The corresponding signifiers of the interface are presented near each other.


• Personalization - The content is shown with screenshots of the internal employee portal, which fosters meaningful learning.

Cognitive Apprenticeship, Scaffolding and Assessment

As the training progresses, learners develop their own knowledge by participating, observing others and by contributing their own scaffolding. An example of that is shown in the video below.

After the demonstration and practice of the response tools, it is expected that: (a) learners use it throughout the course of the training, and (b) learners implement the tool when they are conducting training using WebEx training. The same practice and scaffolding serves as an assessment, where the instructor verifies knowledge and skill acquisition around the tool.

Tension Between Theory And Practice - Tradeoff

During the first iteration of the training, the delivery and implementation of the course as designed posed a major challenge. The instructor struggled with delivering the training seamlessly at the same time that it was catering the needs of different trainees. The challenges were around the technical setup, the level of access and participation of the trainees via technology. Some trainees’ connection was poor, some showed very little skill in finding their way around the different tools and some couldn’t unmute, while the instructor did not know the learners’ individual technical setup. That situation hindered the flow of the training since the instructor had to stop teaching to address technical issues that training participants were facing during class.

Considering that the design for this course is using and Agile, SAM approach, we implemented a new way of delivering the training. We added a role to the training, named Co-Host/Producer. We decided to add a second person to assist the instructor in addressing any technical and learner issues that are not related to learning. Furthermore, after delivering the training with the help of a co-host, we decided to add the use of a co-host to the training content as well. From that point on, every WebEx^® Training workshop is conducted by two hosts, one for delivering the learning, and another to provide technical support and address any non-learning related issues during the course of the workshops. The tradeoff is that the training now requires two people.

This is an example of how the SAM approach helps improve the training on the go as iterations of the training implementation occur.

Final Analysis and Recommendations

The end result of the implementation of this training is the use of a new medium that impacts the way we conduct training at NYU Langone Health. Taking into consideration the urgent and sudden need for the training, all decisions made throughout the design, implementation, evaluation and iteration of the three are sound and effective.

I support my assertion that design decisions were made correctly based on quantitative and qualitative data received both formally and informally.

• Our Learning Analytics (unfortunately, I’m not allowed to show that information) show a steady increase in the use of the WebEx^® Training tool over WebEx^® Meeting tool, suggesting that awareness and know-how of the software have increased. Visits to the support page and downloads of the job aids have also increased, suggesting that learnners are embracing the use of this online tool and also have interest in the job aids and additional performance support.


• On the qualitative data side, my department conducted surveys regarding the use of the different tools provided for working remotely, and early data (data collection is not complete as of this writing) also suggest that employees are favoring delivering and taking trainings using WebEx^® Training over WebEx Meeting.

An important and valuable action that the we take during the trainings is live feedback of the training after each session. We use that information to make adjustments and improvements in the different parts of the Instructional Design as we see them fit.

Reflection

A former VP of my department at NYU Langone used to say: “There are no training emergencies”. I can now say that his assertion was wrong. The “Using Webex^® Training Virtual Platform” training emerged out of necessity. Prior to the pandemic, our primary method of delivering training was in-person, and due to a pandemic, all training must be online. There is a wealth of mandatory and continuing education that is required by regulatory state and federal entities in order for the institution to continue functioning. Additionally, during a pandemic, clinical and non-clinical staff have to be trained on how to function when an emergency, such as a pandemic, emerges.

Since the implementation of WebEx^® Training is part of an emergency transition to virtual settings, at the present moment there aren’t any prerequisites for the training; learners are welcome into the training without any verification that their current technology setup. We offered the training to all staff immediately, and learners came to the training sessions as per their or their directors' request. However, for future and ongoing trainings, I recommend to establish technical requirements for learners workstations as preparation before taking the training. I also recommend to create a checklist of knowledge requirements for learners before class that can help minimize technical issues. Having those prerequisites can reduce the work of the co-host.

Considering the urgent nature of the need for this training, its design approach, implementation and further improvement as iterations take place, seem like the correct process within these circumstances. We ourselves did not know all the features of WebEx^® Training. We had some experience using it since we were in the process of evaluating it for future use and alternatives to in-person training. In my fellow Instructional Designer’s words: “I had to first study and learn on my own the essential functions of the Webex^® Training platform. From that knowledge acquisition, I developed job aids to teach with, researched and found online tutorials from Cisco and YouTube, then set out sharing these resources with my team members.”

I found the following parts of the project successful:

• Agile approach to the design – There just wasn’t any room for use of ADDIE, there wasn’t any use for a formal needs assessment or formal evaluations. Additionally, the iterative nature of SAM and the practical approach of Backward Design are appropriate for this situation.


• Qualitative and quantitative data collection – Not all collected data is directly referred to this training. However, the collected learning-related data suggests an increase in use and improvement in the quality of training using this tool.


• The andragogical nature of the design principles and values – The training offers real-world solutions to the trainers. Also, the raining serves the needs of the organization at the same time that provides comfort and relief to the immediate needs of the trainers.


• The just-in-time approach – The training contains only what is needed for the learners to use the content immediately.


• Asynchronous availability of the material – Recordings of the most recent sessions (the ones with older versions of the software or methods that are improved are archived), job aids and employee portal workspaces provides access to the necessary information and tools to learners as they put what is learned into practice.


• Collaborative and experiential learning – The use of live scaffolding and live demonstrations provide the opportunity of learning by doing.


• The adaptability of the designer – The design has evolved as job aids are added, changes in the interface are released by Cisco, and we integrate learners' feedback.

I found room for improvement in the project in the following areas:

• Prerequisites – As of this writing, all learners are welcome into the live training workshop without any verification of their connection meeting technical requirements. That has caused problems to the point that they decided to add a second person to the design of the training delivery with the purpose of designating that second person to address all technical and other non-learning related needs during training.


• Advanced organizers – Just like technical requirements, an advanced organizer in the form of a simple outline can help learners anticipate what they are going to be learning, along with reinforcing the importance of using WebEx^® Training over WebEx^® Meeting. Some employees are still using WebEx^® Meeting for training out of unawareness of the added features that WebEx^® Training provides. An advanced organizer added to the invitation to the training can help visualize the usefulness of this training.


• Train, not sell – The reality is that there are several challenges in the software interface. Although Cisco has made improvements in some of them, there are still problems with finding certain tools (i.e., at one point the command for sharing computer audio was under “Video” in the menu). Being upfront about the challenges of the software can provide learners with a clear idea of what to expect when conducting training using the tool.


• UDL – There haven’t been any considerations for any of the principles of Universal Design for Learning, more specifically, addressing accessibility.

Through the months in which the training has taken place, there have been several adjustments to the design of the training (in addition to having and recommending a co-host). Some are based on iterations of the training, and some are due to changes in the WebEx^® Training interface by Cisco. These changes have improved the learning experience to learners and increased the level of access to performance support:

• Co-hosting – Having host or producer was found to be a best practice, as it allows the presenter to facilitate without pausing to troubleshoot technical issues for attendees.


• New tools – Enabling the annotating tools on the platform, allows participates to practice using them as an attendees would, and explore the usefulness of this tool for their own training designs.


• Improved practice of teaching the tool – As the instructor herself taught the software repetitively, she became more confident at the same time found better ways of delivering the training, ways that she then integrated in the content and design of the training:
• As facilitator, having a quiet space, with reliable WiFi and broadband service is crucial.


• Use of any hands-free audio device, such as headset or speakers.

Closely studying this training has provided me the opportunity of implementing agile instructional design and experience the effectiveness of implementation and design going hand by hand as iterations of the training happen. Coming from a rigid ADDIE mindset, I was able to better understand (and accept) other approaches to instructional design as it plays a major role in addressing immediate needs in an emergency.

Yes, there are training emergencies.