Adding Entities

Always save a new Document before adding Entities. This is to enable any possible existing Entities in the Document content to be identified via Suggested Entities.

The following methods can be used to add Entities to the Document. The Document must be in EditClick Edit from the upper Editing menu mode to add Entities.

• Suggested Entities
• Entity Extraction
• Search for an existing Entity to add
• Add Entity manually

Note: If there is no detail showing in the Content pane other than this message "Unable to extract content of the file. Check that it represented as text (not image) and has a correct format" the source document used for the content was an image and the text content could not be extracted. (The Document is saved and can be viewed on the Attachments page). In this case the Entity detail would have to be manually added into the Document Content and saved to be able to check for suggested or existing Entities or to Extract Entities. Or you can simply add and search for Entitles manually.

Suggested Entities are Entities that already exist within ModalConnect and have been automatically identifiedby an exact match to their Remembered Terms within the Document content (once saved). It is recommended to review these prior to adding new Entities to prevent Entity duplication and save time.

Review Suggested Entity

When a new Document has been saved, click 'View Suggested Entities' to loadIt can take 1-2 minutes as the background checks are made before all Suggested Entities are identified any Suggested Entities (shown in the lower pane under the Entities list).

To highlight or find a Suggested Entity in the Document content, click on the Entity label. The Entity's remembered terms are highlighted green.

A Suggested Entity can be reviewed and directly added to the Document's Entities list (if it is determined as the same Entity that appears in the Document being viewed). If a Suggested Entity is added to the Document it includes all the existing Entity's Properties and attributes.

To review a Suggested Entity, with the Document in Edit mode click on 'View Details' of the Suggested Entity. This opens the existing Entity's Properties record and provides 2 options;

1. Add to Document - If on review the Suggested Entity is determined as the same Entity that appears in the Document Content - select 'Add to Document' to add the existing Entity to the Entities list of the (current) Document. Additional properties can be added (prior to clicking 'Add to Document) by selecting these from the 'Add Property' list and completing the Property value\detail.
2. Unsure - If selected, the Entity details window is closed and the Entity will remain in the Suggested Entities list and can be reviewed at a later time.

To view a Suggested Entity's Profile right click on the Entity's name or Title from the Suggested Entity list and select 'Go to Entity Profile' (opens in a new tab).

Note: Suggested Entities are identified from the existing Entity's Remembered Terms.

New Entities can be easily created by extracting the details directly from the Document Content. This function is called 'Entity Extraction'. Entity Extraction improves accuracy (by limiting data entry errors), speeds up the process of adding new Entities and automatically creates remembered terms (which provide the required details to identify Suggested Entities).
An existing Entity search occurs automatically during the Entity Extraction process. The Search is made against Entity Properties.

To extract Entity information and search for an existing Entity or create a new Entity follow these steps.

1. Extract Entity detail: Highlight the Entity detail within the Document Content
   
2. Right (oppositional) click on the highlighted text and mouse\hover over the Add Entity pop-up to show the list of available Entity Types.
   
3. Select the required Entity Type
   
4. The 'Add Entity' window opens (over the top of the Document content - this window can be dragged around the Entity Extraction page). The highlighted text creates the search terms and populates the Search field where an automatic search starts against existing Entities of the Type selected.
   Note: the Add Entity window can be dragged around the Document pagedon't drag it of the Document page if working with 2 screens. Cannot also edit or scroll the Document without closing the Add Entity window.

5. Search Results

   If the automatic search finds no results that match all of the search terms with an existing Entity's properties - a pop-up shows with the options to select;
   Yes - to create a new Entity - which opens the New Entity window with the extracted text.
   No - which closes the pop-up but keeps the Add Entity window open where the search terms can be modified for further searches.
   

   Modify Search Terms

   If a Search has returned no results or to explore other possible results, you can modify the terms by removing, modifying or adding search terms and\or applying search conditions. After modifying the search terms either click the Search button or 'Enter' from your keyboard.

   Tip: Before modifying the extracted Search terms from the search box, copy them so that these can be pasted back in if no Entity is found and used for adding to the Entity's properties if you need to create a new Entity.

   

   Entity Search Results

   If Entities are found that match all of the search terms against an existing Entity's properties, the Entity results are listed in the left pane of the Add Entity window.
   If the Entity has already been added to the current Document the text shows green - otherwise results are in grey text.
   

   If Potential Duplicates are found - see this article Potential Duplicate Entities Found
   

   Review Entity Results

   Click on a listed Entity to review their existing Properties in the right pane.
   

   Add Existing Entity to Document

   To add an existing Entity from the search results to the Document, click on the Entity to select it, then click 'Add to Document'.
   
   

   Entity not Found
   

   If following a modified search an existing Entity has not been found there are 2 options -

   1. Select the 'Create New Entity' button at the top of the top of the window (the current search terms in the search field will be available for the 'Selected Document Text').
   2. In the Search field enter or paste (if previously copied) in the original search termsthis then provides the extracted details (selected text) from the document content to add to the Entity's properties and click Search or Enter. From the No results pop-up select Yes to create the new Entity.
      

   Create a New Entity

   Where either the 'Create New Entity' button is selected or 'Yes' from the 'No Search Results Found' pop-up the 'Add Entity' window (for the Type) opens with the Selected Document Text on the left.
   From the 'Selected Document Text' in the left pane, click on content and Drag & Drop onto the relevant Property or enter the value detail.
   The Properties that show when the window first opens are the identifiers used for the Entity Types label.
   Further Properties can be added (where available for the type) by clicking the 'Add Property' button and selecting from the Property list.
   
   As Properties are completed the details are also added to the Remembered Terms (at the bottom of the New Entity window).
   Review the remembered terms and limit these to key identifiers - e.g a unique ID and or Names.
   Learn more about Remembered Terms.
   

   Add Entity to Document

   Once the property details and remembered terms have been completed, click 'Add to Document' and the Entity is added to the Documents Entities.
   

Entities can also be added manually if not using the Entity Extraction method. From the right side pane, click on 'New Entity' then select the required Entity Type from the drop-down list.

This opens the New Entity pane. In the Search field enter the Entity's details and click Search or Enter.
If the Entity is found in the results it can be selected and added to the Document.
If the Entity search produces no results, then select Yes to open the 'New Entity' window

1. Drag & Drop selected Text into the appropriate Property
2. Add and complete further Properties (as appropriate)
3. Review the Remembered Terms and edit if required to keep concise
4. Click Save and Finish
   

Remembered Terms are used to help identify existing Entities as Suggested Entities and play a valuable role to help prevent Entity duplication.
When creating a new Entity using selected textFrom document content and search terms, when content is added to a Property it is also added to the 'Remembered Terms- phrase' at the bottom of the New Entity window. When the Entity is saved/added to the Document if that Remembered term phrase appears in any other Document it can be identified as a 'Suggested Entity'.

The Remembered Terms are editable. It is recommended to remove any characters copied to Remembered Terms and to keep the remembered term phrase concise using main Names or Titles or unique identifiers. The order of the terms is also referenced for identification.
Example: In Document 1 - Entity 'Peter Person Wellington' detail was extracted from the document content and a new Entity created with the Remembered Term phrase saved as 'Peter Person Wellington'

In Document 2 the content contains the Remembered Term - 'Peter Person Wellington' which can now be identified and shown as a 'Suggested Entity' in the Document Entities pane.

If the remembered term is not an exact match or appears as separate text within the Content then it cannot be identified - e.g 'Peter Person of 13 High Street Wellington' because the text 'Wellington' does not immediately come after 'Peter Person' the Remembered Term phrase 'Peter Person Wellington' is not an exact match and will not be identified.

Characters, where used are treated as part of the remembered term - e.g in this example Peter Person (Wellington) would not be identified as a suggested Entity as with the brackets it does not exactly match the Remembered Terms. If the remembered term was simply Peter Person, the Entity would be identified.

Tip: Keep the remembered terms concise by using a unique IDClient\Person ID, Vehicle registration, Driver Licence or Passport number if available or the Entity identifier properties -e.g First Name and Last Name, Organisation name\title, Vehicle registration number etc. The property combinations & order used for remembered terms should be those that Entities would typically appear in Documents for the Organisation. E.G if the Documents loaded into ModalConnect typically show Person Entities by First name followed by Last name, then the Remembered terms phrase should follow that format.

Think of the Remembered terms as a phrase - so to identify a Suggested Entity, that exact phrase (and not just parts of it) needs to appear in the Document content.

Once a Document has been saved, if an Entity's Remembered Terms are detected in the (saved) Content of other Documents, then the Entity can be identified and will show as Suggested Entities which helps prevent Entity duplication.

Note - where Entities have been merged there may be more than 1 Remembered Terms record

Any Potential Duplicates found in search results are existing Entities (of the same Entity Type) that have specific properties of the same value (content) with other Entities. The specific duplicate properties are usually key identifying propertiesThese properties are managed in the Administration tables. such as names, birth-date, IDs, Driver licence, registration, phone number etc. These properties have been assignedin Settings a number value (the Duplicate value) and when the sum of these duplicate properties - that match those of an existing Entity meet a specified threshold, this triggers the identification of a potential duplicate Entity. The duplicate score is automatically evaluated by the system and when potential duplicates are identified this provides an opportunity to review those duplicates and if determined they are a genuine duplicate of an existing Entity they can be merged. It also provides the opportunity to 'tidy-up' potential duplicates that have been identified due to inaccurate details (that may have created 'false' duplicates e.g assignment of an incorrect ID, birth-date etc).

When potential duplicate Entities are detected in the Add Entity Search Results, a Merge Duplicates message displays advising that potential duplicates have been found and provides the option to review and merge duplicates.

Select Yes (or select View Duplicates from the top of this window) and any Potential Duplicates found in the search results are listed in the top pane of the 'Merge Checked Duplicates' window.

Select No to continue reviewing existing Entities search results to add to the Document.

Review Potential Duplicates

To review an Entity's potential duplicates, in the top pane click on an Entity to select - this is the 'master Entity'. In the bottom pane the master Entity is always listed in the tile on the left followed by its potential duplicates.

Tip: The Add Entity window can be expanded by clicking the expand button which allows more potential Entities to be viewed.

• Any potential duplicates of the Master Entity are listed in the tiles to the right ready for review.
• Results can be navigated using the left and right arrows to view the next or previous results.
• The Entity identifier details are shown above their properties, hover over the identifier or use the horizontal scroll bar at the bottom of the tile to view all details.
• At the top of each tile there is a 'View Profile' link (opens the Entity's Profile in a separate tab) so an Entity Profile can be easily accessed for a more detailed review prior to merging.
• The Master Entity's Duplicate PropertiesThese properties have been assigned a duplicate value (in the Entity Type settings) are highlighted blue and appear at the top of the Entity's properties list making these easier to identify and view.
• The duplicate properties of potential duplicates are highlighted blue, where those values match the master Entity's duplicate properties.
• Potential duplicates are ranked and displayed based on the highest score of their duplicate property values.
• Non duplicate-matching properties are listed under the highlighted properties and not highlighted.
• The non matching duplicate properties list can be collapsed by selecting 'Hide Details' and reopened by selecting 'View All'.
• To select a potential duplicate for Merging to the master Entity - check the Merge checkbox (the merge won't be finalised until the 'Merge Checked Entities' button is selected.
• The tiles of potential duplicates selected for merge are also highlighted with a dark border to provide easy identification and are listed first in front of any unchecked potential duplicates. Potential duplicates checked for merging are listed in the order they are selected.
• Multiple potential duplicates can beIf on review you have confirmed that they are legitimate duplicates. Note - a merge cannot be undone. selected (checked) to include in the merge, so you can process 1 or 2 at a time or all checked duplicates.
• To remove a potential duplicate from the checked for Merge list - uncheck the Merge checkbox and it will be moved to the end of the list of unchecked potential duplicates and appear just before any dismissed potential duplicates.
• When reviewing potential duplicates, rather than totally removing a potential duplicate from the results, it can be dismissed by clicking the X in the top right of the tile which sends it to the very end of the list. Dismissed potential duplicates show a yellow border around the tile with the X removed.
• Dismissed potential duplicates can subsequently be reviewed and selected to merge by checking the Merge checkbox which will move the Entity to the end of the 'Merge' potential duplicates listing.

Merge Duplicates

To Merge the potential duplicates that have been checked (to merge), from the top left of the window click Merge Checked Entities then to confirm the merge click Yes on the pop-up. Once the Entities are merged, either continue to review the existing potential duplicates, or select another master Entity to review their potential duplicates, or click Cancel to return to the Add Entity search results.

Caution: An Entity merge cannot be undone - so ensure Entities are reviewed and confirmed as genuine duplicates before clicking Merge Checked Entities.

Once merged all the potential duplicates that were selected are merged with the master Entity. This includes all summary content, notes, tags, propertiesproperties of the same type and value are merged not replicated. E.g 3 Entities merged with the last Name Smith - only shows once. , documents, links, attachments and files.

Post merge tidy-up

Where merged Entities have the same properties but different values, these are added to the merged Entity and may need reviewing to tidy-up e.g edit or delete any unwanted properties. E.g a person showing 2 birth-dates or IDs when they should only have 1 - in this scenario edit or delete the incorrect property. This can be done in a related the Document on the Entity Extraction page (edit Entity) or within the Entity Profile.

To include any Entities found in the initial search results that have a similar Title but are not an exact match to be identified as Potential Duplicates, those Entities would need their relevant properties (that identify a Duplicate value) to be updated within a related Document or the Entity's Profile.

Learn more about managing Entity Duplicates in the File Entity Duplicates