Crawl technology is a valuable tool for content curation. Download this poster, presented by Synterex at the 2022 American Medical Writers Association (AMWA) conference, or scroll down for a text-only version.
Background
An issue that managers often face when onboarding and coordinating resources is having an abundance of data and information and not knowing how to collate or transfer it in an effective way. A “crawl” is the process of using a script or program to collect all content from one or more web pages into a dataset. We will outline the journey of implementing and optimizing crawl technology that leverages artificial intelligence (AI) to curate content and enhance the digital employee experience.
Methods
In Q4 2021, our staff was engaged in discussions about streamlining the employee onboarding experience. We defined what types of information represent the most common inquiries from new employees, in particular entry-level medical writers. The top categories were disease-specific information, regulation- or document-specific information, and client-specific information.
Around the same time, we became aware of the Viva Topics software through our regularly scheduled review of new technologies from the annual Microsoft Ignite conference in 2021. Viva Topics is a Microsoft technology (Redmond, WA) that leverages AI, Office 365 search capabilities, Microsoft’s application programming interface, and various core services (e.g., SharePoint, OneDrive) to put data together and enhance the discovery experience. We selected Viva Topics for these abilities in particular:
- Documenting who is the best resource for a specific therapeutic area
- Having a concise glimpse into a client and their portfolio
- Identifying subject matter experts (SMEs) across technologies, applications, and clients or departments
- Learning more about what topics matter to users
In addition, we liked that in Viva Topics the SME could further update the information in the case of inaccurate results and maintain the information over time, as well as the relative cost-effectiveness of the tool (around $50 USD per user per year). A topic is defined as any concept that is considered organizationally important. The breadth of the crawl is customizable by the software administrators; when deploying Viva Topics, the administrators selected for it to crawl all of the company’s SharePoint sites.
Results
Topic Discovery and Curation
We assigned a “Knowledge Manager” on our Medical Writing Operations team and assigned them permissions to work with the Information Technology (IT) lead and other SMEs to further curate topics, including the following tasks:
- Confirm AI-generated topics (“suggested topics”) as valid
- Reject topics to prevent the content from being viewed as a topic
- Create topics that were not discovered by AI or identify topics that could be edited to be more helpful or accurate
The initial topics AI discovered were mostly client- and disease-specific information. The majority of these terms were confirmed to be topics of interest for the company; only 3 terms were rejected (see Table 1 in the downloadable PDF linked above).
Once these initial terms were confirmed, the AI continued to discover new topics over the course of the month. None of these topics was rejected, and most of them required curation (manual edits to improve the quality of the result, identify the proper SMEs, add detailed descriptions, or link additional files or sites).
User Interaction with Topics
When a topic is created, there are 3 main ways in which the user can interact with the topic (see Figure 1 in the downloadable PDF linked above).
From these 3 points of origin, users can select the topic to learn more about it through the topic details and can even connect with the SMEs directly via chat. Content permissions on sites are set based on the user group in which the user is listed in Microsoft Office, so while the user may be able to see a term, they may have to obtain access to it to interact with it further on a SharePoint site or inside of an Office application.
Conclusions
Viva Topics is a relatively inexpensive product that makes crawl technology that leverages AI accessible to a wider variety of users than has been possible previously. To maximize the investment in crawl technology, plan to have a Knowledge Manager who has broad knowledge across the organization, to understand how the topics interrelate and curate the topics.
Collated topics may need minor maintenance over time to ensure they stay up to date with current regulations and staff; the silver lining is that the initial exercise of refining the topics will be time well spent, as AI will continue to provide suggestions to improve topics as changes occur in your environment. Users can also interact with the information and note whether the description was useful, and AI uses this information to report to administrators and refine descriptions and topic highlighting.
Resources
Microsoft Viva Topics: https://docs.microsoft.com/en-us/viva/topics/topic-experiences-overview
