How AI Can Transform the Content Business

As the promise and potential of AI continues to reveal itself, so do its flaws. While off-theshelf, consumer tools like ChatGPT may be great for whipping up quick emails or updating website copy, its capacity to provide inaccurate results, known as hallucinations, can create unacceptable risk in higher stakes situations, from business to science, where reliable information is of critical importance. This is especially true in the complex world of civil engineering, where the success—and the safety—of any project relies on standards and codes being followed to the letter.

‍

How AI Could Help the ASCE

The American Society of Civil Engineers (ASCE) represents more than 150,000 members of the civil engineering profession, across 177 countries. Through the expertise of its active members, ASCE is a leading organizer of technical and professional conferences and provider of continuing education. As the world’s largest publisher of civil engineering content, they’re an authoritative source for codes and standards that protect the public.  

ASCE came to Mod Op Strategic Consulting (MSC) with a challenge. The organization sells subscriptions to its portfolio of domain-specific civil engineering standards, which are a critical resource for practicing civil engineers to ensure the safety, reliability, and compliance of their projects. Subscribers are able to search through the standards in the domains they are subscribed to; however the current search engine lacks natural language capabilities and requires a degree of understanding of how standards are structured and the terminology used.

Even more critically, there is often valuable information for users to be found in related topics to which the user may not be subscribed. The system presently has no way of identifying these relationships and surfacing the relevant information. This results in users getting incomplete answers, or worse, ASCE losing out on the opportunity to cross-sell and upsell subscriptions to the related standards.  

Finally, one of the biggest pain points for the organization is searches that do not yield the answers users are looking for, resulting in support queries to ASCE’s help desk for one-onone assistance. This strains the capacity of the organization, and causes dissatisfaction

‍

MSC Takes on the Challenge

We hypothesized that AI could provide a tremendous value to ASCE and to its end users, enabling ASCE to:

• Provide 24/7/365 question and answer support
• Improve the speed of support, with questions answered almost instantly
• Reduce the time spent by employees responding to questions
• Free up employees to redeploy their expertise creating additional value
• Provide more complete and accurate answers to subscribers, drawn from the full set of ASCE content resources
• Cross-sell and upsell relevant products to users
• Enrich the capabilities of the current structured search by enabling natural language queries

The caveat: there is, quite simply, no room for mistakes when someone turns to ASCE for tools and advice. This made the incorporation of current off-the-shelf AI solutions impossible.

‍

Digging Into the Details

‍

To better understand and improve the user experience, ASCE presented us with standards surrounding topics from wind loads to ground shaking, along with several frequently asked question types, such as, “What is the difference between a Site Class A and Site Class C soil classification as related to seismic hazards?” and “What information can help me design a building with shutters that might incur high wind loads?”

We then engaged with cognitive AI platform partner CogniSwitch, which is a backend platform that can help tailor AI systems to incorporate complex sources of knowledge, organizing the information in a way that can make it useful for AI systems and enable reliable, hallucination-free responses.

Using CogniSwitch’s large language model (LLM) tools, MSC ran the ASCE content through the process of data ingestion, curation, and provisioning, turning the raw material into a curated knowledge graph that made the dense material functional for AI. We reviewed the results and ensured accuracy and completeness.

We also dug into the data to determine the potential time savings of the project. Using an in-house data scientist and manual sampling with a series of questions, we estimated that a well-functioning AI engine could save a minimum of 2,000 man-hours for ASCE.

‍

Putting the Information to Work

Organizing the data was only part of the challenge. MSC tested a number of user interface possibilities and chose to access CogniSwitch’s APIs through a Microsoft Teams AI Chatbot, rather than build an interface from scratch. This saved time and cost for ASCE without compromising the goals of their project. By connecting the AI chatbot to the organized data, we were able to create a system that could handle even the most complex questions from users.

We tested the chatbot using sample questions provided by ASCE, as well as our own errorf illed alternatives to see if we could “fool” the bot into hallucinating. We asked questions about soil classifications that did not exist — and the chatbot responded appropriately. We asked outlandish questions, such as “what is the best type of car to drive during an earthquake?” and the bot again held the line. Despite our best efforts, we could not fool the bot, and the system consistently generated accurate, helpful answers.

‍

Next Steps

With our successful demo behind us, we built a high-level set of focus areas for our next steps in planning a limited use case pilot, and beyond. Working within ASCE’s budget, we created a roadmap that would allow the organization to test whether an AI-powered system would increase user satisfaction and enable new monetization possibilities. By successfully executing the cutting-edge task of integrating AI with their core business, ASCE can serve their users in new and forward-thinking ways that will take their existing content offering to the next level.

‍