Law firms and legal departments have known for decades that there was gold to be mined from their legal document corpus, but getting at that data was a tedious, manual process. The juice wasn’t worth the squeeze, as they say. But that has all changed with the advent of new capabilities unlocked by generative AI. Leveraging AI to extract valuable data from legal documents has become instrumental in enhancing efficiency and precision, as well as enabling workflows, data analysis, and knowledge capture.

However, the success of many different types of AI projects relies heavily on the quality and structure of your organization’s data foundation, as well as understanding what you want to extract and for what purpose. There is work that has to be done prior to unleashing AI-enabled data extraction to ensure that what you’re getting is valuable to drive your specific processes and analysis needs.

Who This Guide is Designed to Help

This guide aims to aid IT, innovation, and knowledge management professionals in law firms and legal departments through key steps to prepare a robust data foundation for AI data extraction implementations and other content-focused AI projects. If you are in a smaller firm or legal department without these roles or the specific expertise in-house, you may want to partner with outside legal AI and automation consultants who do. Many NetDocuments partners have been engaged in these types of data transformation projects for years, so we would be happy to make those connections for you.

‍

Step 1: Strategic Alignment

Before beginning any AI project (or any IT project for that matter), you need to ensure that your efforts align with the firm or department’s overall strategies and goals. This is the only way to ensure that your project will be meaningful and have the necessary support and resource allocation from the organization’s leadership.

Due to the urgency of most law firms to get meaningful AI projects up and running to drive competitive advantage and law departments’ desire to cover more work in-house, it may be necessary to hire help – whether that is hiring additional internal resources or working with an outside partner. Either of these approaches will require backing and funding from your organization’s leadership.

Finally, getting clear on why you wish to extract data and how it will be used will serve as your north star and ensure that your projects stay within a clearly defined scope that has meaning for the business.

HELP TO GET STARTED: Check out these NetDocuments’ customer stories on how these firms tied their AI efforts to their firm’s strategy:

·         Buchanan on Building an AI Strategy for Legal Automation | NetDocuments

·         AI Apps Help Improve Client Outcomes at Kelley Drye & Warren LLP

Step 2: Data Assessment and Mapping

Before working with your document content, you have to know where it is located, what it consists of, and which systems and people need access to it. To leverage AI to extract data, the extraction tool you are using will also need to have secure access to that data.

Many have already completed data mapping exercises for security and governance purposes, but if you haven’t done that, this would be a good place to start. Identify and catalog all data sources, including digital archives and third-party databases, as well as their integrations.

•  Identify Data Sources: Start by identifying all data, storage locations, and data sources. This is critical for preventing data loss, upholding data accuracy, and determining the correct data involved in processes and analysis.
• Classify Data: Firms and departments need to understand exactly where client data and other sensitive information resides, who has access, and how it’s processed. This data classification allows organizations to improve their data management practices, as well as apply appropriate safeguards and access controls.
• Map Data Flows: Identify and map how data flows within and between systems. Going through the mapping process helps prevent errors or data bottlenecks from happening with extraction processes.
• Ongoing Updates and Documentation: Data maps are dynamic, not static. They require consistent maintenance, updates, and changes when new data sources and tools are added or changed. This will ensure that your efforts do not go to waste and keep data maps up to date as they evolve.

Step: 3: Analyze Risks and Secure Data

Because of the highly sensitive data handled by law firms and legal departments, as well as the numerous ethical and regulatory requirements like GDPR, CCPA, and HIPAA, it’s especially important to review and analyze risks related to the collected data and ensure safe data management practices are being applied.

• Data Encryption: Ensure that data is encrypted both at rest and in transit to protect it from unauthorized access. Additionally, using tools that are available within existing systems and do not require moving data will improve workflows and reduce information governance concerns and the risk of sensitive data being exposed.
• User Access Controls: Implement stringent access controls to ensure only authorized personnel can access or modify the data.
• Software Access Controls: Be sure to review different sources of data to ensure that all third-party app data integrations are able to keep sensitive data safe, especially with new AI tools.
• Compliance: Ensure that your data practices and those of your vendors comply with relevant regulations such as GDPR, CCPA, and other legal standards.

HELP TO GET STARTED:

 Regarding data security, for any new AI tools, in addition to ensuring that your inputs and outputs are not being used to train the Large Language Model (LLM), you will also want to ask your AI vendors whether your content is subject to sensitive content or abuse monitoring by either humans or machines. This is a common practice by AI providers and the types of data handled by lawyers will frequently trigger monitoring systems, which is why NetDocuments sought and received an extra exclusion from Azure OpenAI to protect our customers’ data.

To help in the process of applying additional security for sensitive data, the ndMAX App Builder may be used to identify PII and PHI contained within your data that is stored in NetDocuments.

Step 4: Data Integration

AI systems and mission critical workflows often require data from various sources to be integrated seamlessly, so this should be part of your data review process.

• Database Management Systems: Use robust database management systems (DBMS) that support seamless integration and cross-referencing of different data sources.
• API Integration: Identify or develop APIs to enable dynamic data interactions between internal databases and third-party tools.
• HELP TO GET STARTED: Tools like NetDocuments open API and Power Automate Connector can be helpful for these types of undertakings where information residing in one system needs to be shared with another.

Step 5: Data Cleansing

For your data extraction project to be successful and not too costly, be sure you’re working with the most current and relevant data.

•  Remove Redundant Data: Eliminate duplicate records to avoid AI tools being misled by repetitive information or incurring extra AI token costs due to processing redundant data.
• Archive Data: Eliminate obsolete content to avoid AI tools being misled by outdated information and the cost of processing old content. If you do not already have a records retention policy, as well as processes for archiving closed matters and retiring the associated data, that is another project that you may wish to undertake to ensure that AI tools only have access to data that is most current and relevant.

HELP TO GET STARTED: If you have a DMS like NetDocuments, it is easy to use the Creation Date or Last Modified Date fields to identify content and matters that could potentially be archived, but you may need to engage someone with records management expertise, if you are not clear on legal records retention requirements.

Organizations like your State Bar Association, ARMA, and the Association of Legal Administrators have resources like these: Law Firm Guide to Document Retention, Establishing the Best Record Retention Policy for Your Law Practice, and Top Record Management Strategies for Law Firms.

Step 6: Understanding Your Data Types and Relevant Content

Before the advent of generative AI, metadata was limited to what you could reasonably expect a busy legal professional to enter into a document profile in your document management system (DMS), if you even had one. Now, it’s all about what generative AI can extract from your files, but generative AI is not magic. It needs guidance from your subject matter experts on what content is important to them to enable workflows, processes, or data analysis.

Legal documentation encompasses various types such as contracts, leases, wills, copyright submissions, pleadings, court rulings, and many more. Before embarking on any type of extraction project, you need to know which types of documents make sense for extraction and what data is important to extract based on your specific practice areas and jurisdictions.

If your firm has a document management system (DMS), the document types might be leveraged as a good starting point, but you may need to do some clean-up or even go a level deeper to identify document sub-types, because there can be many different types of agreements or pleadings whose data extraction needs to be slightly different. For example, an employment agreement contains very different information from a purchase agreement, and the associated workflows are quite different as well.

To optimize AI data extraction:

• Start Small: Work with your legal teams to identify what content is important to them to enable workflows or for data analysis. Focus on one practice group who is interested and willing to participate fully and then build on their successes to drive your project forward. For example, something as simple as a pleading caption generator may seem small, but because it is repeated numerous times over a litigation matter’s lifecycle, it can be a huge timesaver.
• Categorize Existing Document Types: Review existing document types by Author or Practice Area and create a comprehensive list categorizing different document types and deciding whether you need to add sub-types to be able to categorize them properly for extraction purposes.
• Determine Extraction Fields: Work with your SMEs to review the unique data fields and determine which ones would be good candidates for data extraction. Instead of having your SMEs start from a blank slate, you can use an AI assistant to identify data fields for specific document types. Remind them that not everything may need to be extracted. Only those things that might be used as part of a workflow or needed for analysis should be candidates for extraction.

HELP TO GET STARTED: At the end of this article are lists of content you might want to extract from litigation pleadings and commercial real estate documents. To generate these lists, I used the prompt, “Within litigation pleadings, what pieces of information would be good to extract to enable litigation workflows, processes, or analysis?” with our ndMAX Legal AI Assistant.

Step 7: Data Standardization and Taxonomies

We can’t repeat it enough: Consistency is key for AI to accurately process data.

Taxonomies are hierarchical classifications that can help optimize data organization and retrieval. Whether they realize it or not, your legal teams already have at least some loose version of a taxonomy in their existing content. Some larger organizations may already have their own fully fleshed out taxonomies or leverage something like the SALI legal taxonomy[MS1] [MS2] , which is also used by legal technology providers.

Here are some best practices to ensure that your data is as useful as possible.

• Custom Taxonomies: Develop taxonomies specific to your law firm's practice areas, ensuring documents are categorized in a way that aligns with your operational needs.
• Cross-Referenced Taxonomies: Implement taxonomies that enable cross-referencing among different document types and practice areas, facilitating comprehensive search and retrieval.
• Updating Taxonomies: Regularly review and update your taxonomies to reflect changes in legal practices, document types, and organizational needs.

Step 8: Build the Metadata Structure

When you do data extraction, the content you extract will become valuable metadata to be leveraged to power workflows or data analysis, so you need a way and place to store that information and enable use in the ways that your SMEs have indicated they need.

• Gather Basic Document Metadata: If you have never had a DMS, you may wish to undertake an initial project to identify basic document metadata before doing a more detailed AI-enabled data extraction project. Basic metadata provides context which is crucial for AI to understand and categorize documents correctly. For example, in your DMS, Practice Area and Doc Type fields can provide needed information to help with your initial AI extraction projects. Establish a clear set of initial metadata fields such as client, matter, author, document type, creation date, and last edit date. Many partners of NetDocuments have existing processes to easily extract these types of basic information.
• Define New Metadata Fields: Next, consider the additional data that you wish to extract from your content and where it will be stored. This will be dictated by: (1) your technology capabilities, (2) how the data will be used and secured, and (3) what data you wish to expose to end users. For example, NetDocuments provides the ability to store metadata as profile fields tied to each document, which means the data is available to anyone with access to the document, or as part of a data table, which resides in the background and has more limited access but can still be used to enable processes.
• Evaluate Metadata Capabilities: Some systems limit the number or type of metadata fields that you can have, so you may have to make some hard choices. With a system like NetDocuments, you will have unlimited, nested and dynamic metadata fields.  This means you will also need to determine metadata relationships and dependencies. For example, if using status fields as part of a knowledge management process, you may have additional fields that are completed or data that is extracted as the process progresses. Again, having detail- and process-oriented SMEs to help is critical to your success.
• Formatting Consistency: Consistency will be key for AI to accurately process data as part of workflows and data analysis going forward after your initial data extraction project. Standardize formats for dates, names, addresses, and other commonly used fields.

Step 9: Ensuring Success

To ensure success, you need to plan for the unexpected and ensure you have adequate testing and feedback loops included.

• Handle Missing Data: Develop strategies to manage missing data, such as using placeholders or imputing values based on logical inference.
• Benchmark Testing: As part of your process, ensure that your SMEs are included to do benchmark testing and confirm that the extracted data is accurate and meets their needs. Be sure that their expectations are realistic. For example, humans are not 100% accurate all the time and AI won’t be either, so you will want to agree on what percentage of accuracy is acceptable at the outset.
• Performance Metrics: It probably goes without saying but define clear performance metrics to measure the success of the project. This includes accuracy, efficiency, and user satisfaction.
• User Feedback Loop: Create a feedback loop where users can correct the AI’s mistakes, improving the model’s accuracy over time.

HELP TO GET STARTED: During testing, it can be helpful to have two people doing the same review. That way, if there is a disagreement with the human reviewers, then it would make sense why the AI might not be able to properly classify something. This can be a good indication that tweaks in the process need to be made for higher accuracy.

Step 10: Plan for the Future

Continuous monitoring and maintenance of the data foundation ensure sustained AI performance in the future.

• Regular Audits: Schedule regular data audits to identify and rectify inconsistencies, changes, or errors.
• Automated Annotation: After your initial data extraction project, you will want to use AI to automate the tagging of documents with relevant metadata during the data intake or creation process, so keep these additional steps in mind as you are defining your initial project.
• Adaptive Tagging Systems: You will also need to consider how to handle metadata tagging as changes occur over time. Implement systems that adapt metadata tags based on changes in the document's content and usage patterns over time.

By laying a solid data foundation, those running projects in law firms and legal departments can significantly enhance the effectiveness and efficiency of AI-powered document data extraction. Proper preparation, continuous monitoring, and adherence to best practices will lead to a transformative impact on how legal documents are managed and utilized, turbo-charging workflows and revolutionizing data analysis capabilities.

HELPFUL RESOURCES

Sample List of Content to Extract from Pleadings

Leverage an AI assistant like the ndMAX Legal AI Assistant to give your SMEs a starting point for which content they would like to have extracted. For this list, we used the following prompt: Within litigation pleadings, what pieces of information would be good to extract to enable litigation workflows, processes, or analysis?

Each of these pieces of information can help streamline case management, improve document organization, enhance legal research, and facilitate strategic analysis for ongoing litigation. When you know the specific pleading types you are working with, you might want to use a more specific prompt to get more detailed information.

• Case Caption: Information related to the parties involved (plaintiff and defendant), jurisdiction, and court.
• Case Number: Unique identifier for the case which is essential for tracking and organizing documents.
• Dates:
  
  • Filing Date
  • Hearing Dates
  • Deadlines for responses, motions, and other case-related milestones.
• Nature of Suit: The type or category of legal issue being addressed.
• Legal Claims and Defenses: A summary of the causes of action or defenses asserted in the pleadings.
• Parties' Attorneys: Names and contact information of the legal representatives for each party.
• Factual Allegations: Key facts that form the basis of the legal claims or defenses.
• Relief Sought: The specific remedies or resolutions being requested by the parties (e.g., damages, injunctions).
• Exhibits and Attachments: Any supporting documents or evidence filed along with the pleadings.
• Court Orders and Rulings: Information on any orders or decisions made by the court throughout the case.
• Motions Filed: Details of any motions filed by the parties, including their outcomes.
• Docket Information: The case’s docket sheet, which lists all filings and court actions in chronological order.

Sample List of Content to Extract from Commercial Real Estate Documents

Leverage an AI assistant like the ndMAX Legal AI Assistant to give your SMEs a starting point for which content they would like to have extracted. For this list, we used the following prompt: Within commercial real estate documents, what pieces of information would be good to extract to enable legal workflows, processes, or analysis?

Extracting and organizing these pieces of information can greatly improve the efficiency and accuracy of legal workflows or analysis related to commercial real estate transactions and disputes. When you know the specific document types you are working with, you might want to use a more specific prompt to get more detailed information.

• Property Details:
  
  • Legal description of the property
  • Physical address
  • Parcel number
  • Size and dimensions of the property
  • Type of property (e.g., office building, retail space, industrial)
• Ownership Information:
  
  • Name(s) of the current owner(s)
  • Ownership structure (e.g., individual, corporation, LLC)
  • Previous ownership history
• Transaction Details:
  
  • Purchase price or lease terms
  • Financing arrangements (e.g., mortgage details, lender information)
  • Dates of important transactions (e.g., purchase date, lease commencement, expiration date)
• Tenant Information:
  
  • Names of tenant(s)
  • Lease terms (e.g., duration, renewal options, rent amount)
  • Responsibilities for repairs and maintenance
• Zoning and Land Use:
  
  • Current zoning classification
  • Permitted uses under the zoning code
  • Any zoning variances or exceptions
• Easements and Encumbrances:
  
  • Details of any easements (e.g., utility easements, access rights)
  • Liens or other encumbrances affecting the property
• Environmental Reports:
  
  • Results of any environmental assessments (e.g., Phase I/II ESA)
  • Known environmental issues or contamination
• Building and Improvement Details:
  
  • Description of structures on the property
  • Condition and age of buildings
  • Details of any recent or planned improvements
• Regulatory and Compliance Information:
  
  • Compliance with local building codes
  • Permits and licenses required for the property use
  • Records of any violations or fines
• Insurance Information:
  
  • Details of insurance policies covering the property
  • Coverage limits and policy terms
• Legal Contracts and Agreements:
  
  • Copies of any lease agreements
  • Joint venture agreements
  • Property management contracts
• Financial Statements and Reports:
  
  • Operating statements
  • Profit and loss statements
  • Rent rolls and occupancy rates
• Dispute History:
  
  • Records of any past or ongoing legal disputes involving the property

‍