This high-level proposal is a direct response to the current advancements in artificial Intelligence technology, specifically Generative Pre-Trained models, and will outline the opportunities, risks and considerations for evaluation.
DISCLAIMER: At the time of writing I am an employee of Cochlear LTD. All information used in this, and related, articles are based on publicly available sources. At no point was privileged or internal information used in any way. This was done as a part of a master's degree assignment.
Overview of analysis & proposal methodology
The methodology for this proposal will follow the formal Technology and Innovation Management (TM) framework. As a discipline, TM has a 50-year history of research and use which offers universally accepted conceptual models to understand and communicate technology innovation management.
The framework emphasises the dynamic nature (Dynamic capabilities theory) of the knowledge flows that must occur between the commercial and technological functions in an organisation, linking the strategy, innovation, and operational processes. (Çetindamar, et al, 2016)
Opportunity Statement of new technology
Machine Learning and Artificial Intelligence have been in development for several decades, however, over the last few years incredible progress in these fields has captured the world’s zeitgeist.
Specifically Large Language Models (LLMs) have reached a stage of practical maturity, in the form of Generative Pre-Trained (GPT) Models, that has allowed them to pass the following exams in the last month (as of February 10th, 2023):
University of Pennsylvania - Wharton MBA
US Medical License
Chartered Financial Analyst (CFA)
Stanford Medical School final for Clinical Reasoning
The University of Minnesota Law final
What makes these achievements most notable is that these were done without any specific or specialised training for these exams, or subjects.
The GPT model was trained on 175 billion tokens of data (~100 billion words) and was able to apply this, without any further changes to the model, to these exams. Also worth noting is that the training data includes these languages: (Kublik, Saboo. 2022).
English (primary)
Spanish
French
Chinese
Russian
Portuguese
Python
JavaScript
C++
C#
Java
Ruby
PHP
Go
To be clear, LLMs are not capable of reasoning as they are statistically probabilistic models to predict the next word in likely order.
However, the emergent capabilities of these large models are sufficiently close to human reasoning to perform several real-world functions at a relatively advanced human level.
It would be reasonable to assume that this technology is currently in the early stage of the growth S-curve and that we are about to experience a significant technological paradigm shift. (Chen, et al. 2012)
Analyse the organisation.
As the market leader in the implanted assisted hearing market, Cochlear has very clear and industry-leading organisational core competencies:
1. Cochlear & Baha Device manufacture
2. Support eco-system around devices – with Clinics, Hospitals, and Audiologists.
3. Sales eco-system around devices – Relationships with medical and governmental bodies across the globe.
4. Medical device research and academic collaboration – Collaborations with several universities and commercially aligned entities, sponsorships of academic entities and scholarships for audiologists.
Prioritisation by Business strategic objectives
By reviewing the Business’s strategic objectives, we can determine the appropriate technology to investigate further. (Cochlear Limited, 2022)
1. Retain Market Leadership.
2. Grow the hearing implant market.
3. Deliver consistent revenue and earnings growth.
Prioritisation by Risk objectives
By understanding the Business’s risks, we can position the appropriate technology. (Cochlear Limited, 2022)
1. Pandemics
2. Product innovation and competition
3. Misappropriation of know-how and intellectual property infringement
4. Medical Device regulations
5. Product Quality
6. Market Access
7. Credit and Currency
8. Interruption to product supply
9. Privacy and Information security
10. Talent management
11. Geo-political risk
Current Status of TM Capabilities in Cochlear
With ~60% market share globally of the Cochlear and Baha (Bone anchored) markets, our technology, products and services are world-class and industry-leading.
Using the TM framework, we will ask and examine:
1. How do we exploit our technology assets?
2. How do we identify future technology that will have an impact on our business?
3. How do we select technology for business benefit?
4. How should we acquire new technology?
5. How can we protect our technology assets?
6. How can we learn from our experience to improve our ability to develop and exploit the value of technology?
TM Activity | Status |
Acquisition Relates to any need for technology acquisition |
|
Exploitation Relates to market penetration, technology robustness and technology comparative advantages. |
|
Identification Relates to the identification of technological opportunities and threats |
|
Learning Relates to technological change impacts and the ability to apply continuous improvement |
|
Protection Relates to IP and technology protections in place, and mechanisms. |
|
Selection Relates to knowledge and implementation of new technological capabilities and innovations |
|
Identification - Potential Use cases of LLMs
Due to GPT models' versatility across tasks that it has not specifically been trained for, there are many potential uses. Listed below are the specific use cases identified that would be most applicable to Cochlears current operations, segmented by enterprise domain. (Kublik, Saboo. 2022).
Some of the systems listed below are compound Machine Learning models, utilising bolt-on LLM technology. (Harvard Business Review. 2021)
Marketing
Marketing content generation, translation, and assistance
Customer Domain – Recipients and Candidates
Customer Service GPT (Connected Care)
Frontline website and apps chat
Audiological and device assistance for recipients
Customer sentiment analysis, keyword extraction, and trend analysis
Organize & summarize product feedback from users (support tickets, surveys, analytics) into actionable insights for future product development, and improved service delivery
Customer Domain - Professionals
Professional Audiological GPT
Training of audiologists
Device reference for clinics and clinicians
Medical reference for all professional users
Enterprise efficiency
Enterprise Internal GPT
Enterprise search,
Internal training support,
Medical and Device reference assistance,
Technical assistance,
Medical texts & Research summarisation,
Summarisation for decision-making
HR Assistance GPT – support, internal sentiment analysis, keyword extraction, and trend analysis
DevOps and Coding GPT - technical assistance, automation and technical artefacts support and generation
Prioritisation decision matrix
LLM technology use-case | Strategic Objectives Alignment | Risk Objectives Alignment | Total Value (Sum of Alignments) |
Marketing | 1,2,3 | 2,5,6 | 6 |
Customer Service GPT | 1,2,3 | 2,4,5,9,10 | 11 |
Professional Audiological GPT | 1,2,3 | 2,3,4,5,10 | 11 |
Enterprise Internal GPT | 3 | 1,2,4,5,9,10 | 7 |
HR Assistance GPT | 3 | 4,5,9,10 | 5 |
DevOps and Coding GPT | 3 | 1,2,4,5,9 | 5 |
The Customer Domain is the priority based on both strategic and risk alignment, with the current service touch points processes summarised below.
Current State
Customer Domain Functions
Responding to customer inquiries: answering customer questions and resolving any issues or concerns they may have.
Order processing: taking customer orders, processing payments, and ensuring that orders are fulfilled in a timely and accurate manner.
Technical support: helping customers with technical issues, such as troubleshooting problems with products or services.
Complaint resolution: receiving and resolving customer complaints, often through a process of investigation, negotiation, and dispute resolution.
Account management: managing customer accounts, including handling account changes, billing inquiries, and payments.
Sales and marketing: promoting products and services to potential customers and helping to close sales.
Escalation management: handling complex customer issues that require a higher level of expertise or attention and ensuring that these issues are resolved in a timely and satisfactory manner.
Feedback and suggestion management: collecting and analysing customer feedback and suggestions and using this information to improve products and services.
Quality assurance: monitoring and evaluating the quality of customer service to ensure that it meets established standards and expectations.
Reporting and analysis: generating reports on customer service metrics and analysing data to identify areas for improvement.
Technology Audit of selected technological domains for LLMs
Gap analysis
Current Process & Technology | Capability gap improvement opportunities |
Web and app The Customer Service department facilitates contact through email, and phone centres. The information is captured in a Customer Relationship Management tool Salesforce. |
|
|
|
|
|
|
|
Training In-person, when and where available |
|
Access to the Cochlear Professional portal which allows access to the knowledge base and technical escalation. |
|
Access to the Cochlear Professional network which allows access to the knowledge base and escalation via the portal or direct contact |
|
Identification - Value Analysis
Value analysis is a process used to evaluate the benefits and costs of a technology to determine its overall value to a business. This process helps to make informed decisions about technology investments and ensure we are getting the maximum return on investment. (Phaal, et al. 2011)
Using GPT technology for the customer domain has the following advantages and disadvantages.
Advantages
Increased efficiency: provide quick and accurate responses to customer queries, reducing response times and increasing overall efficiency.
Improved accuracy: By fine-tuning GPTs for specific use cases, high levels of accuracy in understanding customer inquiries and providing appropriate responses can be achieved.
Cost savings: reduce labour costs associated with having human agents handle customer inquiries.
24/7 availability: beneficial with a global customer base or for customers who require support outside of traditional business hours.
Enhanced customer experience: provide customers with more personalized and engaging experiences, improving overall customer satisfaction and loyalty.
Disadvantages
Limited empathy: Unlike human agents, GPT technology lacks the ability to provide emotional support and empathy to customers, which can impact the quality of the customer experience.
Limited understanding: GPT technology may not be able to fully understand the nuances of certain customer inquiries or may provide incorrect responses, leading to customer frustration.
Inflexibility: GPT technology is limited by the data it was trained on and may not be able to adapt to new or unique customer inquiries without additional training and fine-tuning.
Technical limitations: GPT technology can be expensive to implement and maintain, and may require specialized technical expertise to operate effectively.
Ethical considerations
Bias – ML Models retain the inherent biases of society and can manifest in operation.
Confabulation – AI “Hallucinations” are the result of probabilistic uncertainties captured in the model’s training.
Saliency - the black box nature of AI can be difficult to position in a heavily regulated market such as Medical Devices
System proposal to close gaps.
Conceptual design of Target Architecture for the first and second releases.
R1 = Stand-alone GPT support for Customer Services Function
R2 = Customer Service function regulating (HFRL) GPT front-line servicing
Release 1
Release 2
Human Feedback Reinforcement Learning (HFRL)
HFRL is a type of machine learning that uses human input to guide the learning process of an AI system. The AI system receives feedback from human operators in the form of rewards or punishments, which are used to adjust the system's behaviour. Human operators provide more nuanced and context-specific feedback on the system's behaviour and are safeguards against Bias and Confabulation, which we covered earlier. The GPT system may receive positive reinforcement when it provides helpful and accurate responses to customer inquiries and negative reinforcement when it provides incorrect or unhelpful responses.
Over time, the GPT system will learn from this feedback and improve its performance, providing a better customer experience.
Value Curve of Service Offerings*
The Value-Curve of our perceived service offerings versus competitors shows that there are several improvement opportunities that implementing GPT technology will help improve.
*Dummy data
Opportunity Cost and Benefit analysis
Opportunity cost refers to the cost of an alternative that must be forgone in order to pursue this technological implementation. It represents the value of the best alternative use of resources and time.
Cost of doing nothing:
- Competitors may implement first.
- Cost of future implementation and catch up maybe significant.
The potential benefit of implementation:
- Higher professional customer satisfaction
- Higher recipient customer satisfaction
- Significant cost savings
- Faster and better device support
- Faster insights and value realisation
- Well positioned for future advances in GPT technology
Forecast the Technology and environment.
In the next few years, the adoption of GPT technology in the medical device industry is expected to experience significant growth.
From streamlining supply chain management to improving product development and regulatory compliance, the potential applications of GPT in the medical device industry are numerous. As the technology continues to evolve, it is likely that more and more companies in this industry will adopt GPT to gain a competitive advantage and meet the ever-increasing demands of the healthcare sector.
This trend is expected to lead to an acceleration in the pace of innovation in the medical device industry, with companies leveraging GPT to develop new products, improve existing ones, and provide better care to patients
Analyse and forecast the Market User
As industries and society embrace GPT technology there will be an expectation from customers for the speedy, in-depth and personalised customer experience that this technology will increasingly become better at.
Acquisition of technology
Technology or Activity | Buy (License) | Make | Collborate |
GPT model | Licensing models for GPTs are increasingly available and cheaper as time progresses. | Expensive and requires unique SMEs outside of Cochlears’ core competencies. Training costs would be in the millions of dollars for each fully trained model. | Possible to do, however, the cost of development and maintenance may be significant |
Integration | Would be provided as part of the AI as a Service (AIaaS) / Platform as a service model (PaaS) | Expensive and requires unique SMEs outside of Cochlears’ core competencies | Achievable and cost-effective as Python skills are not prohibitively expensive. Skills exist in-house and would need up-scaling. |
Support | Would be provided as part of the AI as a Service (AIaaS) / Platform as a service model (PaaS) | Expensive and requires unique SMEs outside of Cochlears’ core competencies | Achievable, however certain aspects of deeper technical support may be expensive. |
Customisation | May be challenging depending on the vendor’s service model. | Full customisation is needed and will require support. | Achievable, not as expensive as making the toolset but more expensive than licensing. |
Implementation - Portfolio Management
The needs of the program of work will need to be balanced against the existing Technology Portfolio, should we move forward. The rank prioritisation will dictate timelines and scope will indicate the budget.
Resources
1. Technical personnel with knowledge of GPT models and tools
2. Technical personnel with knowledge of tool integration
3. Hardware and software resources to implement the GPT model, tools, and integrations.
4. Access to any necessary external services.
a. Contactors
b. SME’s
Implementation Road Map and Stage Gates
Proposal by Releases to mitigate any potential issues and provide as safe as is possible implementation.
R1 = Stand-alone GPT support for Customer Services Function
R2 = Customer Service function regulating GPT front-line servicing
1. Preparation: Establish the plan for the technological and operational implementation of the LLM GPT model.
2. Requirements: Determine the necessary technical requirements and resources to implement the model against Stage Gate One requirement. Cost out Licensing vs Collaboration approaches.
3. Selection: Select the GPT model and tools that best suit the organization's needs.
Stage-Gate One – Stakeholder reviews of the vendor demo system.
- Edge-cases
- Robustness
- Bias
- Saliency
- Confabulation
- Regulatory compliance
- Disaster processes
4. Governance, Quality and Operational Model design finalisation: all processes, quality and governance considerations will require significant workshops and collaboration across several functions. This should be led by the IT Architecture team.
5. Training: Train the team on how to use the GPT model and associated tools.
6. Implementation: Install and configure the GPT model and tools.
7. Testing: Test the GPT model and tools to ensure they are working correctly.
Stage-Gate Two – Technical and Operational field-testing validation.
Roll out of the system in one geographical domain, to run in parallel with existing systems. Hyper-care model to facilitate HFRL and any hotfixes as required.
8. Deployment and Change Management: Deploy the GPT model and tools to the remaining geographies in a methodical manner, with extensive change management support.
9. Continuous Improvement / Continuous Deployment (CI/CD) and Kaizen: As the technology teams are rolling off the main development roadmap, we will create support and CI/CD teams to continually improve the functionality. Human Feedback Reinforcement Learning (HFRL) – feedback and analysis of requests and outputs by customer service, medical and device experts will also assist to facilitate continuous improvement of the system.
Key Performance Indicators (KPIs)
How we will measure the success of the technological implementation:
Benchmarking against current processes (will require a Time Analysis study) baselines.
Cost savings annualised against existing spend for identified Customer Domain functions
Comparisons against best-in-class, cross-industry, implementations of this technology in the customer domain
Total Quality Management (TQM) – questionnaires and surveys
Protection
Depending on the final acquisition model we will implement a legal IP review process.
Identifying and measuring technology assets (internal and external)
How best to manage knowledge workers and their IP
IP portfolio management
References
3 Areas Where AI Will Boost Your Competitive Advantage. (2021). Harvard Business Review. https://hbr.org/2021/12/3-areas-where-ai-will-boost-your-competitive-advantage
Bennink. (2020). Understanding and Managing Responsible Innovation. Philosophy of Management, 19(3), 317–348. https://link-springer-
Çetindamar, Phaal, R., & Probert, D. (2016). Technology management activities and tools (2nd ed.). Macmillan Education.
Cochlear LTD. (2022). Cochlear Annual Report 2022. Corporation Listings. https://www.listcorp.com/asx/coh/cochlear-limited/news/2022-annual-report-2749627.html
Chen, C.-J., Huang, Y.-F. and Lin, B.-W. (2012) ‘How Firms Innovate through R&D Internationalization? An S-curve Hypothesis’, Research Policy, 41(9), 1544–1554.
Kublik, & Saboo, S. (2022). GPT-3 : building innovative NLP products using large language models. O’Reilly Media, Inc.
Phaal, R., O’Sullivan, E., Routley, M., Ford, S. and Probert, D. (2011) ‘A Framework for Mapping Industrial Emergence,’ Technological Forecasting and Social Change
Tamkin, A. (2021, February 4). Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. arXiv.org. https://arxiv.org/abs/2102.02503
Technology Organisations, management and innovation. (2020, April). The Open University. https://www.open.edu/openlearn/nature-environment/organisations-environmental-management-and-innovation/content-section-1.7
Villegas, C. (2018a, April 6). Roger Pen/ Roger EasyPen (For Cochlear Implants). Arizona Hearing Center. https://www.azhear.com/roger-pen-roger-easypen-for-cochlear-implants/
Villegas, C. (2018b, April 12). Streaming Technology (For Cochlear Implants). Arizona Hearing Center. https://www.azhear.com/streaming-technology-for-cochlear-implants/
Witzel, L. (2023, February 1). ChatGPT and DevOps: Practical Uses for Disruptive AI | The TIBCO Blog. The TIBCO Blog. https://www.tibco.com/blog/2023/01/12/chatgpt-and-devops-practical-uses-for-disruptive-ai/
Comments