Week 3 Update

In the past two weeks, we have interviewed 20 beneficiaires which include a mix of primary and secondary data users, as well as higher ups(generals and commanders). Besides, we had the very precious opportunity to meet with general Dempsey, the 18th Chairman of the Joint Chiefs of Staff and the 37th Chief of Staff of the Army, and general McChrystal, the joint Special Operations Command in the mid-2000s. We have gained a broader view of how military adopt technology from these two prestigious military leaders. Moreover, we have visited Fort Bragg on 1/29, which we gained hands-on experience on current in-placed tracking devices and data analytics tools.

 

  • BENEFICIARIES

 

 

We organized our beneficiaries in three main groups listed below:

 

Primary Data Users

  1. Cognitive Performance Coaches
  2. Data Analysts
  3. Research Psychologists

Secondary Data Users

  1. Physical Therapists
  2. Spiritual Advisors
  3. Cognitive Performance Coaches
  4. Interpersonal Coaches
  5. Strength Trainers
  6. Dietitians
  7. Other Sports Medicine Specialists

 

Higher Ups (General and Support Staff at SWCS)

  1. General Santiago
  2. Support Staff/ Board that drives downstream policy changes.

 

  • INTERVIEWS & KEY TAKEAWAYS

 

 

Name Title Email Takeaway Interviewer
Hector Agayuo SWEG Command Sergeant Major aguayoh@socom.mil Lack of bandwidth of personnel to collect data
He looks outliers or errors (compare that with the actual situation)
Which means he need to know each soldiers quite well, how would he be able to do that?
He goes through 6 months of CCC data in 2-3 hours (110 surveys)

NC
Greg Santiago Operations Specialist, SWEG gregorio.santiago@socom.mil Logistic & operation planning does not involve a lot of data related to our project

Same biometrics data may means different things as each individual is unique. How to assess data can be much more difficult than how to collect them  

BX
MAJ Chuck Schemacher; Operations Officer, SWEG charles.schumacher@socom.mil Is SWIC measuring the right stuff? Why do many students who pass Assessment and Selection drop out later on?

Brass want more graduates, cadre want better quality graduates.

New generation of soldiers are different to train, causes friction.

Bob Jones Communications Language School Director HDP operates here to evaluate the trainees effectiveness at negotiation NC+BX
James Arp HDP Director james.arp@socom.mil Funding comes from:

USSOCOM, USASOC, Army

We have to justify the value of the training programs to the 2,3, and 4 star general commands

NC
General McChyrstal  (x2) he joint Special Operations Command in the mid-2000s Must change the culture as well as the technology within the military All
Mike Taylor HHC 1st Sergeant michael.s.taylo More robust data collection/analyis would help determine what training interventions work and what does not.

cooperation/information sharing is generally facilitated by relationships, if they aren’t there, then generally different units won’t communicate.

AJ
General Dempsey the 18th Chairman of the Joint Chiefs of Staff and the 37th Chief of Staff of the Army Need to focus on what problem we can solve

Technology is changing the landscape of the federal gov and DoD

Focus on the cultural aspect of getting military

All
COL Bill Rice SWEG Commander william.rice@socom.mil Higher-ups at SWEG don’t have much contact with the committees.

SWEG doesn’t have a committee that makes data-driven decisions.

Peer feedback, instructor feedback, and some data informs human performance results.

30 day ideal innovation turnover for new devices. He would like to be able to test new hardware/software without jumping through the regulation/ permission hoops.

Use case:

Immediate feedback mechanism as the top use case priority from Commander Rice point of view. Have expert as a reference point for novices

using big data/database to make key decision such as selection process/recruiting. Track each trainee’s performance/data throughout the training process

BX
Major Chuck Is SWIC measuring the right stuff? Why do many students who pass Assessment and Selection drop out later on?

Brass want more graduates, cadre want better quality graduates.

New generation of soldiers are different to train, causes friction.

Stephen M. Mannino, Edd Human performance program coordinator stephen.m.mannind@socom.mil Showed the database that strength and conditioning coaches use to advise

Strength and conditioning coaches have to download to share the database with other data users (physical therapists, CPC’s, data analysts, research psychologists)

All
Justin Jones Strengthen conditioning coach justin,jones@socom.mil Names are declassified – only coaches have access to the names

Database’s real time feedback has been helpful in readjusting physical training

All
Taylor McKinney Physical therapist Accessing the medical database always leads to problems

No protocol for how physical therapists make data-informed decisions

All
Sara Butler Physical therapist Would really like a database that they can get to without taking up so much time

Did not interact much with CPC’s, data analysts, and research psychologist

All
Alexandra Hanson Data Analyst alexandra.hanson.ctr@socom.mil Getting buy-in from higher ups is necessary to help increase funding and implementation

Acquisition for different accounts at SWEG is siloed which means that different solutions will not arrive at the same time.

Constance Garcia Data Manager constance.garcia.ctr@socom.mil Government shutdowns affects contractors.
Daniel Gajewski Performance Integrator Daniel.gajewski@socom.mil The Toby eye tracking device just arrived and they are still working on calibrating it

The current output of that eye tracking

device is just a video

Has another software platform for further analysis but they haven’t start yet

All
Dawne Edmonds Process Improvement and Project Management

US Army Special Operations Command

Dawne.edmonds@socom.mil There’s no authoritative data source. It means that there is not a single data source that the analysts rely and trust upon.

“CONSTANTLY. I CAN’T EXPLAIN TO YOU HOW OFTEN IT HAPPENS”

They tend to type all the data all over again. This is a constant problem. Army are multi-service.

NC
Oscar Gonzalez Research Psychologist oscar.gonzalez@usuhs.edu Is in the process of holding CPC’s more accountable for inputting and collecting data

Working on concerns from data analysts about the large amount of time spent toward data input

All

 

 

  • KEY INSIGHTS

 

 

  • Data utilization is actually quite low. The decision making at the committee level are mostly based on intuition and past experience
    • Many data (both biometrics and cognitive) are not collected due to lack of tracking devices
    • Even for those collected data, most of them are not being utilized
  • Most data are self-reported in the form of survey or self-assessment, which involves basis
  • Most database are not shared internally, many secondary data users work in silo and do not have access to each other’s database.

 

 

  • KEY PROBLEMS

 

 

  1. Secondary users have three or four different data systems for one function
  2. Secondary users have to get data manually
  3. There is no central database for the data from biometric devices
  4. Secondary data users work in silo
  • Hard for secondary data users to collaborate with each other
  • Hard for secondary data users to collaborate with primary data users
  1. Lack of a systematic workflow among primary data users
  • Some CPC collect and scrub data, some don’t
  • CPC does not have a standard process or expectation on data-related work
  1. Errors in data collection (ex: bear incident)
  2. No plan for biometric devices and what outcomes they should have
  3. Training feedback
  • Soldiers do not receive real-time feedback
  • Soldiers do not receive specific feedback on training performance
  1. Who gets access to which data
  2. The regular maintenance of the system & platform and periodic update

 

 

  • KEY DECISIONS

 

 

  1. We need to define the scope of the problem and narrow down the problems
  2. Define the key pain point and focus on the corresponding problem
  3. Re-identify key beneficiaries based on the problem
  4. Identify use cases for the re-identified key beneficiaries
  5. Prioritize use case
  6. Start brainstorming potential solution for each use case

 

Week 2 Update

Insights

This week we began by better understanding the Seymour Johnson AFB Pharmacy’s general workflow and solidified our understanding of the issue at hand. We also began to present our MVP concept to our beneficiaries better align our solution concepts to needs of the consumers.

 

We quickly realized that our MVP from last week could better account for our beneficiaries’ wants and needs. First and foremost, we realized that a web-based, or even an app-based MVP, would pose too high of a barrier for patient beneficiaries to access. The technological literacy and hardware requirements were too high for the large proportion of patients who were retirees in their 60’s and 70’s. In addition, we also learned that the long wait times cause issues among waiting patients and requires significant amounts of manpower be used in addressing them. Taking this information into account, we modified our MVP: lowering our technological barrier to access and increasing the amount of communication and transparency between the pharmacy staff and patients. This updated MVP addresses the problem as we currently know it while also accounting for the traits and habits of our beneficiaries.

 

Beneficiaries:

[USAF]

Major Villalonga

– Issues worsened with move to new location with new computer systems and equipment.

– Issues include understaffing, the GSL system (is slow and difficult to teach volunteers to use), 7 different isolated computer systems, and small lobby (vs separate pharmacies for refills, AD/dependents, and others)

– GSL system helps to reduce medication errors by volunteers dispensing incorrect medications

 

Pharmacy Tech: SSGT Wray

– Rotate roles every 2 hours, alternating job types without repeating blocks; used to have a scanner, but retirees required guidance

– Challenges: understaffed (finally fully staffed in the last 3 months); IT problems (e.g. CHCS crashing, which prolongs delays, and each program requiring 2-3 monitors; inputting data is very slow); volunteers; new systems (CHCS and GSL are new since the move vs only Pharmassist before); officers are pulled to other duties/meetings elsewhere; some enlisted/NCOs do not show up for duty; GSL’s texting feature was not purchased

– Ideas: chatbot/automated texts

 

[Patients]

SSgt Ashley Torres (active duty)

– SJ has the longest wait time she has experienced; shortest 1h, longest 2h; bases overseas did not have veterans and had automated systems for patients to scan their tags

– Ideas: drive-through pharmacy for veterans and (Android) app to notify when med ready for pickup (vs the wait time)

 

David Gurley (retiree)

– Challenges: parking at the new location is worse than before and a much longer walk; e-prescriptions from on-base prescribers do not always make it to the pharmacy.

– Aware that meds can be mailed, but fills on-base because of habit, socialization, and it gives him an excuse to be on base again. The cost is also a barrier to some.

– Ideas: auto-SMS prescription status; explicitly/graphically depict the front-end pharmacy flow for patients so they know where to go and in what order; drive-through window like retail pharmacies; make information about busy/lull times available

 

TSgt Myron McElroy (active duty)

– Challenges: medications are often not ordered and Helicare (prescriber) system is unstable, so there is a mix of e-Rx and paper Rx, though the latter has been refused for filling at least once; check-in kiosk not reliable; even when the kiosk works, patients still have to check in at the window

– Ideas: notification when Rx is received and when med(s) is ready; [Android or iPad] app share information about (average) wait time

 

Joseph Wooten (Retiree)
– Experience: Fills meds 2-4pm, usually waits ~30 minutes; is understanding of active duty patients getting priority; would not get meds mailed because it requires using a computer, but would be ok with the cost for meds for which he already pays a copay
– To improve: informing patients of changes in estimated pickup dates when the pharmacy runs out of medications BEFORE patients show up at the pharmacy and wait in line to pick them up
– Ideas: automated phone call or mail

 

[Multiple roles]

Donna Martin (Patient and Volunteer)

– The check-in kiosk not working, window scanners not working, difficulty using (non-GSL) computer systems, and line extending to the clinic contribute to the wait time; patients also sometimes cut in line

– She has difficulty with the computer systems and with some people being notified that their meds are ready for pickup when they are actually not

– Many volunteers are new since the location move, as experienced ones quit

– Ideas: kiosk allowing selection of refill vs new fill (like at other bases); separate pharmacies for retirees and AD/families; scan ID into kiosk and verify ID on screen; automated text/call when prescription ready for pickup

 

[External]

Duke Outpatient Pharmacy Tech – Georgette

– Usual wait time for new Rx is 15-20 minutes

– Pharmacy well-manned with experienced staff who rotate duties daily and take the initiative to help smooth bottlenecks in flow.

– Pts can call in for refills – default 24h turnaround time; prescriptions are 90% electronic (rarely need to be fixed/edited), 8% hard copy (entered manually), and 2% faxed (entered manually).

– A limited number of people are assigned to fill controlled substances; counted twice; dispenses are additionally logged manually for accountability.

– New Rx: tech enters/verifies Pt info in system -> drug picked and scanned in to verify the NDC -> pills counted -> expiration date checked -> to pharmacist for verification of Pt info and sig

– Refills: tech prints a label, counts pills, and verifies the NDC; refills do not require any pharmacist intervention

 

Ashley Smith (Civ Pharmacy Technician) and Shannon Brewer (Civ Pharmacist)

– Adopted retail practice of patients lining up to only turn in prescriptions – takes ~5 minutes; active duty get priority (red bins); on-base prescriptions (white bins) filled before those received from off-base (usually handwritten Rx; yellow bins); different queue for Rx to be picked up the next day; separate lines for active duty and patients with disabilities; they have a drive-through open until 7pm; used to use QFlow, but now calculate wait times by hand

– Challenges: staffing levels have remained constant, though the base’s population has grown

– Patients prefer to drop Rx off and return later – no difference in their wait time

– Initiatives: trying to implement secure messaging so that people can queue through email instead of phone

 

Rachel Fefer (Mentor)

– She has TriCare, notes that army bases in DC do not have this issue, wonders why Seymour Johnson AFB does.

– She has many contacts who are experts in the military-healthcare field, including one at CRS who is an expert on military healthcare/TriCare

– Once we clarify what we aim to do, reconnect with her

 

 

Week 1 Update

We talked to several people to get a better scope of the issue at hand, and came away with a few key insights.

Beneficiaries

USAF

TSgt Christina LaRiccia | 940-923-4219

  • Emphasized the necessity of Red Cross Volunteers for the pharmacy to function despite issues
  • Pharmacy is regularly understaffed, especially at peak hours (lunch)
  • Lack of experienced employees/volunteers, especially since new retirement rules implemented

TSgt Perez | 201-233-1577

  • Manages QFlow system, calculates wait time and displays for patients
  • Some volunteers are unable or unwilling to use QFlow causing delays, also sometimes issues with other systems
  • Clinic has 4 non-integrated systems running simultaneously (AHLTA, CHCS, QFlow, GSL)

Volunteers

Hazel Huffman | 937-499-4151

  • Volunteers at pharmacy and manages, hires Red Cross Volunteers
  • Notes confusion between separated windows for refill, filling prescriptions
  • Notes the lack of manpower behind the counter, and feels that hiring more volunteers/training the current volunteers would help solve the issue

Jay Killian | 919-988-1005

  • Volunteer who spends time prepacking, distributing medication at windows
  • Noted the lack of synchronicity between GSL, CHCS computer systems
  • Frustrated over fluctuating wait times throughout the day despite notifying patients of shorter wait times at earlier hours

Patients

Melvin Hudgins Sr, SFC | 919-920-4379

  • Retired former Army Officer, worked in logistics and supply management at SJAFB
  • Frustrated with long wait times, especially when active duty service members are prioritized
  • Recommends dedicated window for active duty, separate windows for retirees/community

Mrs. Nailor Thompson | 706-718-6656

  • Wait times consistently >1 hour, even at off-peak hours in morning/evening
  • Duties are rotated hourly for volunteers, which can disrupt the flow of work
  • Patient flow is an issue, with confusing lines and people all over

Vendor

Shelton Louie (GSL)

  • GSL products were designed to maximize patient safety, automate rote tasks, eliminate shrink loss/theft, and allow non-pharmacists to conduct minor tasks
  • RFID cabinets maximize accountability and reduces picking/bundling errors
  • Several big name clients in the military and civilian space

External

Walgreens – Brad M. | 919-433-6255

  • Biggest bottlenecks are in contacting insurance, financial info
  • Fills emergency orders first, the rest are filled based on pick up time

Rite-Aid – Ryon Chao | 919-389-5341

  • Transfer to new computer system has created major disruptions to workflow
  • Major issue is Pharm Tech training, especially in transcribing phone calls to computer
  • Recommends imposing hierarchy of roles, but not strict distinction (ie Primary phone handler, then a secondary handler in case the first is busy)

Insights

The problem sheet we received from the Air Force at the end of last semester seemed to indicate issues with the workflow, physical layout, and technical (e.g. UI/UX issues) with the software and hardware the pharmacy staff used behind the counter. As we progressed through the beneficiary discovery phase, we began to realize the problem had significantly changed since Seymour Johnson AFB had submitted the problem to H4D. Though some of the pharmacy’s issues had been resolved with time in the new facility, a number of new issues have also sprouted up since then. Some of these issues had to do with chronic understaffing of the pharmacy team, new systems, new workflow, and a high proportion of retired patients coming from off-base clinics.

Decisions

At the end of this week, we are determined to expand our list of beneficiary contacts. We intend to speak with more of the same people, such as pharmacy techs and patients. However, we also plan on speaking with new categories of people, as well. We intend on interviewing the vendors who provide the pharmacy’s hardware and software systems to better understand their products and the role their products play in the workflow. We would also like to better understand how these products work in cohesion and the staff’s experience using them. We also intend on speaking with base leadership to better understand our capacity to affect change in the workflow and workspace. Next steps also include obtaining a better understanding of how the pharmacy’s multiple, siloed systems interact and play into the workflow, either through video or other visual aid. This will give insight into treatable inefficiencies in the workflow, if there are any. Now that we have an updated and better grasp on the problem at hand, we will also begin asking beneficiaries, such as patients and techs, about their sentiments on a product with the capabilities of our MVP.