User's Guide: Avails Validator

Release 0.9

Contents:

  1. Overview
    2.
    1. Functionality
      2.
    2. User Interface
      3.
  2. Using the Validator
    3.
    1. Selecting Processing Options
      2.
    2. Running the Validator and Interpreting Output
      3.
    3. Editing XML
      4.
      1. Overview
        2.
      2. Launching the Editor
        3.
      3. User Interface
        4.
      4. Saving Changes and Re-Validating
        5.
    4. Converting XLSX to XML
      5.
  3. Running Scripts
    4.
    1. Functionality
      2.
    2. Writing Command Scripts
      3.
    3. User Interface
      4.

1) Overview

1.1) Functionality

The Validator is a standalone application that is intended to be used during the pre-processing of an Avails Metadata file. Capabilities include:

  • Validates that one or more Avails files are in conformance with the Avails specification.
  • Validation of Avails file in either XML or XLSX format.
  • Optional checks for conformance with applicable best practices or similar guidance.
  • Integrated XML Editor with linkage to any errors or warnings detected.
  • Conversion of XLSX formatted Avails into XML.
  • Ability to run in a scripted mode for batch processing of multiple files.
  • Installers for Windows, Mac OSX, and Linux platforms.

1.2) User Interface

Figure 1: Validator UI

The user interface is shown in Figure 1. From top to bottom, the main components are:
  • Menus providing access to generic capabilities. The primary one is the "Processing" menu. This allows the user to select the types of validation checks to be performed and the level of logging output desired.
  • The Tool Bar, which is used for:
    • selection of one or more files to be validated, and
    • access to the integrated Manifest Editor.
  • The Logging panel used to display and filter the messages resulting from the validation process

2) Using the Validator

2.1) Selecting Processing Options

  1. The first step is to configure basic options and preferences (e.g., logging output). This is done via the check boxes in the Processing menu (Fig 2).

    2. Figure 2: Validation Criteria Menu

  2. The next step is to select which files will be validated. There are two mechanisms provided:
    • A left-click on the 'File' text area (to the right of the button) will pop open a File Chooser dialog. If a directory is selected, all Avails files found in the directory will be processed. Alternatively a single file may be selected. NOTE: if the selected source is a directory containing multiple Avails files, then all files will be validated against the same set of options
    • The File -> Recent.. menu item may be used to select from a history of recently accessed files.

2.2) Running the Validator and Interpreting Output

Once all validation options have been set, the validation process may be executed via the RUN button. When the validation process completes, the results will be displayed in the Logging panel (see Figure 3).

Figure 3: Validation Results

The navigation pane on the left side allows the user to select a sub-set of messages to be displayed on the right side. Messages are hierarchically grouped first by the Avails file they apply to, then by the severity level, and finally by the type of validation check resulting in the message (e.g., CMD). The levels, in order of decreasing severity, are:
  • Fatal : Indicates the inability to perform any processing on a file. For example, attempting to process an Avails based on an unsupported version of the schema would result in a FATAL log entry.
  • Error : a problem that must be corrected. An example would be the absence of a required property or the use of an unrecognized value for an enumerated type (e.g., an invalid language code)
  • Warning : a problem or inconsistency that does not have to be corrected but may be worth looking into. For example, a Transaction with more than one SuppressionLiftDate terms is contradictory but may still be processable by a retailer.
  • Notice : indicates that a recommended Best practice was not followed (e.g. the use of non-EIDR ID values for IDs)
  • Debug : diagnostic output that should only be produced when requested by Movie Labs.

The last type of log entry is an Info message. These are used to provide a user with summary and status information about the file being validated. The generation of Info messages may be enabled or disabled via the Processing -> Logging menu (see Fig 2 above).

Icons are used to indicate the most severe problem found in any branch of the message hierarchy. For example, Figure 3b shows that only one of the two files that were processed contains errors.

Figure 3b: Severity of Log Messages

The message display pane on the right provides a table showing all the messages in which ever branch of the message hierarchy has been selected in the navigation pane. Some log messages will provide additional information via tool tips. Where possible, this will include references to specific sections of the documentation that cover the identified problem.

The interpretation of the Line entry will depend on the type of Avails file being processed. For XML files it will be the line number in the file while for XLSX files it is actually the row number in the sheet being processed.

The displayed log messages may be sorted by means of a left-click on any of the column headers. Subsequent left-clicks will reverse the sort order.

By right-clicking on the message display pane, a pop-up menu may be accessed (see Figure 4). Available actions are:

  • show the XML associated with the error in an Editor window [for XML-formatted Avails files only],
  • clear the entire contents of the message log, or
  • save to a file the currently displayed group of messages in CSV format

Figure 4: Message Log Pop-up Menu

The saving and/or clearing of the log file are simple operations and the user interface for both of these is self-explanatory. Usage of the Editor is covered in detail in the next section.

2.3) Editing [XML files only]

2.3.1) Overview

The Avails Validator includes a built-in XML Editor that may be used to correct any errors found during the validation process. Note that this is not intended to provide a full-featured XML authoring capability similar to those provided by tools such as XML Spy or Oxygen-XML. Rather it is focused on providing an easy-to-use and integrated capability for quickly correcting errors and then re-running the validation tests.

The Editor UI is implemented as a separate window (see Figure 5). Multiple Editor windows may be opened at the same time, with each displaying a different XML file. Note, however, that there may only be one Editor window for any given file.

Figure 5: XML Editor

2.3.2) Launching the Editor

As described in Section 2..2, an Editor may be launched by the context menu displayed when the user right-clicks on a log entry. Selecting the Show in Editor item will cause an Editor for the associated file to be displayed. The location (i.e., line) in the file associated with the log entry will become the current line and will therefore be indicated by the line number being highlighted in red. For example, in Figure 5, the Editor window shown was accessed by clicking on a log entry linked to line 611 in Avails file test02.xml .

An Editor may also be launched using the button in the tool bar: .

When an editor is launched for a specific Avails:

  • if an editor window for the file has not been previously accessed, a new Editor will be created and displayed.
  • if an editor for the file already exists but has been minimized (i.e., iconified) by the user, it will be opened and brought to the foreground.
  • if an editor for the file already exists, it will scroll to display the selected line in that file.
  • if the log message used to access the editor does not have a specific location associated with it, or the editor was launched using the Menu Bar, the current line will be set to the first line in the file.

2.3.3) User Interface

The Editor window has the following components:

  • a text pane which uses colors for syntax highlighting,
  • a menu bar providing access to the available editing functions
  • line numbers displayed on the left edge that includes markers indicating errors and warnings, and
  • display of status messages across the bottom of the editor window.
These are shown in Figure 6.

Figure 6: Editor UI

The status messages are linked to the marker icons displayed along side the line numbers. A single left or right click on a marker will result in the associated log entry being displayed in the status panel.

2.3.4) Saving Changes and Re-Validating

To save changes, simply use either the pull down menu or the <Ctrl>S shortcut. Once the Avails file has been saved the XML may be re-validated.

The changed Avails may also be saved under a new name using the Save As menu option. If the Avails is saved under a new name then the user must explicitly identify the new Avails file as the Validator's Source before it can be validated. Also it is important to remember that the existing Editor window will still be displaying the original Avails file. In order to edit the newly saved file, the user must:

  1. select the new file as the source file to validate,
  2. run the validation, and then
  3. select one of the resulting log messages to open an Editor window for the new XML file.

2.4) Conversion to XML [XLSX files only]

Whenever an XLSX formatted Avails file is processed, an XML version of the same information will automatically be generated and saved. The XML file will be saved in the same directory as the Excel file it was generated from. It is important to remember that the generated XML may not be valid in that any problems or issues present in the XLSX will also be present in the XML version.

3) Running Scripts

3.1) Functionality

Validation of multiple Avails files may be performed using scripts that can specify any of same options and settings available thru the user interface. A script may easily be used to process a set of manifests with specified sub-sets being handled using different options. All results are automatically displayed in the GUI's log display but the script engine supports the option of also writing the log to a file.

3.2) Writing Command Scripts

3.2.1) Structure

A script file is written using the JSON syntax. The structure for a validation script is as follows:

  • Command: should be specified as validate . The validation command has OPTIONAL sub-elements followed by a list of files to process.
    • Options in the following sub-elements will apply to all files that will be processed. These are:

      • pathPrefix : [OPTIONAL] a root folder for locating all Avails files. The path may be absolute or relative. If relative, the location of the script file is the starting point for resolving the full path to each file or folder. If a pathPrefix is not specified, the present working directory will be used as the starting point for resolving the full path to each file or folder.

      • logging :

        • level : [OPTIONAL] the minimum level of messages to be included in the output log. Allowed values are:
          { "Verbose", "Warning", "Error", "Info"}
          The default when not specified is Warning .

        • output : [OPTIONAL] file to write the log to. If not specified, the log is not saved. The path may be absolute or relative. If relative, the location of the script file is the starting point for resolving the full path.

    • A set of XML-formatted Avails files to be processed. These are specified as a JSON Array. Properties are:

      • file : [REQUIRED] Path to a either a file or a directory containing one or more files. If relative, the starting point for resolving the full path is the pathPrefix .

3.2.2) Examples

The following is an example of script that validates two files plus any additional Avails found in a specific folder and then saves the message log in a single output file:

{  
  "validate":{  
    "pathPrefix":"./AvailsDemos/",
    "logging":{  
      "level":"Warning",
      "output":"./scriptLog01.xml"
      },
    "avails":[  
      {  
        "file":"si_tests/CMM_BBB_v0.1.xml" 
      },
      {  
        "file":"si_tests/CMM_Sita_v0.1.xml" 
      },
      {  
        "file":"studioFoo" 
      }
      ]
    }
}

The next two examples illustrate the use, or non-use, of the pathPrefix . In the first example, all relative paths are resolved using the present working directory as the base. Both foo1.xml and foo2.xml should therefore be in the pwd . 

{  
  "validate":{   
    "avails":[
      {  
        "file":"foo1.xml" 
      },
      {  
        "file":"./foo2.xml" 
      },
      {  
        "file":"/foo3.xml" 
      }
    ]
  }
}

In the next example, foo1.xml and foo2.xml are still co-located but are now in in a sub-directory of the directory containing the script file. foo3.xml should still be in the same location as the previous examples since it is specified using an absolute path. 

{  
  "validate":{   
    "pathPrefix":"./AvailsDemos/",
    "avails":[
      {  
        "file":"foo1.xml" 
      },
      {  
        "file":"./foo2.xml" 
      },
      {  
        "file":"/foo3.xml" 
      }
    ]
  }
}

3.3) User Interface

Selection and execution of a script is via the Run Script.. menu entry. Script files are not selectable using the File text-entry field used to specify an Avails XML file.

Once a script file has been selected the execution is automatic. It is recommended that the contents of the log display be cleared prior to executing a script. Failure to do so will result in any previous messages being included in the saved log file.

Figure 7: Script Selection and Execution