How-to Guide: Reporting

reports-intro1.jpg

 

Overview

The Reports tab provides access to all data collected from an activity. Reports can be viewed in aggregate or on an individual activity basis. Each activity includes a standard set of reports, in addition to any customer-specific custom reports. All reported data is displayed in real time and many reports include an option to export the data to Excel.

To access an individual activity’s reports, begin by selecting an activity from the Manage Your Program menu, then click the Reports tab. 

Reporting is also available in aggregate by selecting All Programs view from the Manage Your Program menu. 

Request for Credit Reports

Overview

eCommerce Reports

Participation Metrics

Overview

ACCME Reports

Download Tab

Live Event Reports

Overview

 



 

 

Aggregate Reporting

How to Access Aggregate Reporting

 

Request for Credit Reports

Request for Credit Reports provide access to data collected from all test(s) and survey(s) in the activity.  In addition, a credit report and any comparison or custom reports will be listed here.



(Back to Table of Contents) 

Test Reports

Each test in the activity will include a standard set of reports.  An export link is also available to download all data collected from the test.

rfc-reports2.jpg

The Test Response Report shows how learners answered each test question.  Counts, percentages, and graphs show the number and percentage of learners who selected each answer choice. reports-response-report-test1.jpg


(Back to Table of Contents)
 

The Test Question Accuracy report displays the percentage of learners who answered each question correctly.  This report is a good way to see if there are any test questions that learners had more difficulty with in relation to the other test questions.

test-reports1.jpg

 

(Back to Table of Contents)
 

The Pivot Reports option allows you to see how learners answered one test question based on their answer to another test question.  To generate the report, select a row and column component, then click the Create button. You can also create pivot reports that pivot across all surveys and tests in the activity

test-reports4.jpg


(Back to Table of Contents)
 

The Test Scores Distribution report shows the counts and percentages of learners who achieved each test score. The bars on the graph show the counts for each test score, while the table below the graph shows the percent of learners who achieved each test score. The average and median scores are also displayed at the top of the report. 

test-reports2.jpg


(Back to Table of Contents)
 

The Average Test Score by Attempt report displays the average score for each attempt at a test.  Each of the bars on the graph represents an attempt number, while the height of the bar corresponds to the average test score of all learners for that attempt.

test-reports3.jpg

(Back to Table of Contents)
 

The Test Score Details report provides the test date, score, and number of attempts for each learner who completed the test.  Clicking a learner’s name, reservation code, or score will allow you to drill down to more detailed information.
test-reports5.jpg

test-reports5-1.jpg


(Back to Table of Contents)

Exporting Reports

Each test and survey in the activity, as well as the Credit Report, contains an Export link to download all data used to compile the report(s).  

rfc-reports3.jpg

rfc-reports3-2.jpg


(Back to Table of Contents)

Survey Reports

Each survey in the activity includes a standard set of reports. An export link is also provided for downloading all data collected in the survey, including learner profile information. 

rfc-reports4.jpg 


The Survey Response Report is in a very similar format to the Test Response Report. This report displays counts and percentages of learners who selected each answer choice. If the report contains rating style questions that use a Likert scale, the average for those rating questions will show in the upper right-hand corner.  For open-ended response questions, click the View Responses link to view each learner’s response. rfc-reports5.jpg


(Back to Table of Contents)

Credit Report

The Credit Report shows the count and percentage of learners who selected each credit type available in the activity, along with the total credit issued. Hover the cursor over the bars in the graph to see exact counts for the credit amount. You can also use the drop-down menu to toggle the report by year. 
rfc-reports7.jpg


(Back to Table of Contents)

Cross Analysis Pivot Report

Use the Cross Analysis Pivot Report to create reports that pivot across all of the surveys and tests associated with the activity.  The report allows you to compare how learners answered one question, based on their answer to another question.  This report is different from the pivot report available for each test and survey in that this report allows you to pivot across all tests and surveys in the activity, not just an individual one.
rfc-reports6.jpg


(Back to Table of Contents)

Participation Metrics

Participation Metrics reports provide information related to Program Attendance, Program Use Statistics, Data Entry Statistics, Fulfillment, Recruitment, and also Geographic Analysis. These reports are available for all activities activity types (Live, Enduring, Journal, Webinar, etc).

Participation Metrics reports are also available in aggregate reporting.

participation-metrics-1.jpg


(Back to Table of Contents)

Program Attendance Reports

Program Attendance reports can help answer questions like:

  • Are there any trends in the daily participation numbers for the activity?
  • Is there a particular state that had above average participation?
  • What professions and specialties were most interested in the activity?
  • Were there any learners participating from other countries?

participation-metrics-2.jpg

The Program Complete by Date report includes 2 charts that show daily and cumulative participation totals. Counts are given for completed, outcomes, and registered (live events only) learners. Hover the cursor over a point on the graph to see the actual counts. You can adjust the timeframe of the report by using the date fields. 
participation-metrics-18.jpg


(Back to Table of Contents)
 

The Program Complete by State report shows activity participation by state. States with the darkest shading have the highest participation counts. Switch views between Completed and Started (includes both completed and in-progress) learners.  You can also hover the cursor over a state on the map to see the actual counts. 
participation-metrics-14.jpg


(Back to Table of Contents)
 

The Program Complete by Country report shows activity participation by state. States with the darkest shading have the highest participation counts. Switch views between Completed and Started (includes both completed and in-progress) learners.  You can also hover the cursor over a country on the map to see the actual counts. 
participation-metrics-17.jpg


(Back to Table of Contents)

The Program Activity by Profession report breaks down activity participation by profession. Counts are given for both Completed and Attended (includes both completed and in-progress) learners. You can sort the graph by either profession name or highest participation count. Hover the cursor over a bar in the graph to see the actual counts. 
participation-metrics-15.jpg


(Back to Table of Contents)

The Program Activity by Specialty report breaks down activity participation by specialty. Counts are given for both Completed and Attended (includes both completed and in-progress) learners. You can sort the graph by either specialty name or highest participation count. Hover the cursor over a bar in the graph to see the actual counts. 
participation-metrics-16.jpg


(Back to Table of Contents)
 

Program Use Statistics

Reports in the Program Use Statistics area can help answer questions such as:

  • Was there a particular point in the activity where learners tended to leave the activity?
  • How many learners started the activity compared to how many finished the activity?
  • How long, on average, did it take learners to complete the activity?
  • Was there a particular day of the week or time of day that participation was highest?

participation-metrics-3.jpg

The Program Workflow Component Visits Count report contains the count and percentage of learners who have visited each component of the activity.  The number of total visits, the number of unique visits, and the percentage of unique visits are listed in the chart. This report is helpful in determining if there is a particular point in the activity where participation levels dropped off significantly.
participation-metrics-7.jpg


(Back to Table of Contents)

The Program Workflow Progress Summary report shows the count and percentage of learners who have completed each component of the activity.  This report answers the question of how many learners started the activity vs. how many finished the activity. It can also be a good report to check and see exactly where the particpation drop-off occured in the activity.  
participation-metrics-8.jpg


(Back to Table of Contents)
 

The Program Use by Day of Week can help identify participation trends related to the day of the week. Hover the cursor over the pie chart to show reveal the actual participation counts. You can also sort the columns. 
participation-metrics-9.jpg 


(Back to Table of Contents)
 

The Program Use by Hour of Day can help identify participation trends by providing a graph showing participation data related to the time of the day.  Hover the cursor over the line graph to see the participation counts for the time of day. (This report is displayed in the local time zone)participation-metrics-10.jpg


(Back to Table of Contents)
  

The Program Use Duration Summary displays the amount of time it took learners to complete the activity.  The average duration, median duration, minimum value, and maximum value are also shown. 
participation-metrics-11.jpg


(Back to Table of Contents)

Data Entry Statistics

The Data Entry Totals report allows you to view how many learner participation records were entered via manual data entry.  The report also lists the name of each data entry account with the total number of records entered.
participation-metrics-13.jpg


(Back to Table of Contents)

Fulfillment Reports

The Program Fulfillment by Date and the Fulfillment Distribution reports provide information related to learners generating an award (certificate, letter of participation, etc) at the end of an activity.  Daily and cumulative fulfillment totals, as well as offline and online fulfillment statistics, are listed in the reports. 
participation-metrics-4.jpg


(Back to Table of Contents)

Recruitment Reports

The Program Recruitment Campaign Summary report tracks how each learner entered the activity.
participation-metrics-12.jpg


(Back to Table of Contents)

Geographic Analysis Reports

Geographic Analysis reports can provide useful insights when marketing your activities to specific geographic regions in the country.  The Default Territory Tree report displays activity participation by geographic region (state, county, city) while the Program Summary by Zip Code report provides activity completion totals by zip code. Both reports allow drill-down to individual learners for each geographic area or zip code. 
participation-metrics-6.jpg


(Back to Table of Contents)
 

eCommerce Reports

eCommerce reports provide detailed financial information for the activity. 

All financial information (sales price, discount codes, net discount, etc.) is included in the Complete Program export on the Download tab.  

ecommerce1.jpg

The Program Sales report provides a breakdown of program sales, packaged program sales, and token sales.  Use the date filter to view data for a specified time period.
ecommerce2.jpg


(Back to Table of Contents)

ACCME Reports

The ACCME Reports tab allows you to export all PARS information for the selected activity.  You can then upload the report directly to the ACCME PARS site. You can also perform a PARS export for ALL activities in your account by using the ACCME PARS export tool from the Aggregate Reports section

Additional activity specific PARS information can be added to each activity from the Program tab.  For more information, please refer to the ACCME PARS How-to Guide.

PARS1.jpg

1.  Begin by first selecting a date from the drop-down menu.
PARS2.jpg

2.  Next, specify the date range for the data in PARS export. For the ACCME, this is expected to be January 1st and December 31st of the activity reporting year.  The start year must also match the end year, as pictured below.
PARS3.jpg

3.  After selecting the date range, click Export to open a new window and save the PARS XML file.
PARS4.jpg

4.  To save the export, right-click on the webpage, then select Save As… to save the file to your computer. BE SURE TO SAVE AS AN XML FILE. After you have saved the file, you may log into the ACCME PARS site and upload the file.

Please note, this is only the PARS report an INDIVIDUAL activity. To generate a PARS report for ALL ACTIVITIES, go the Aggregate Reports view
PARS5.jpg

The message at the top of the above image that reads, “This XML file does not appear to have any…” is not an error. The message is displayed by the browser because the page content is shown in XML format (required by PARS) for you to download, instead of HTML.

When uploading the XML file, the ACCME PARS site will automatically add to or modify existing information that may have been previously uploaded to the PARS site.  Existing information that was uploaded to the PARS site will not be duplicated, only modified as needed.


(Back to Table of Contents)

Download tab

The Complete Program Results export provides a CSV downloadable file that captures all data collected from the activity. All data collected from the learner, including profile data, test/survey data, credit information, eCommerce information, and more is included in the export.
download.jpg

 The following information is contained in this report:

  • Participation ID
  • User ID
  • Code
  • Prefix
  • FName
  • MI
  • LName
  • Gender
  • Birth Date
  • Email
  • Company
  • Job Title
  • Address Line 1
  • Address Line 2
  • City
  • State
  • Zip
  • Country
  • Phone Area Code
  • Phone Number
  • Fax Area Code
  • Fax Number
  • Custom1
  • Custom2
  • Notes
  • Award Date
  • Complete Date
  • Last View Date
  • Expiration Date
  • Recruitment Campaign Id
  • Recruitment Campaign
  • Profession
  • Specialty
  • Specialty2
  • Degree
  • Degree2
  • Designation
  • Client Group
  • Origin URL
  • Site code
  • NPI Number
  • ABMS Membership (Primary)
  • ABMS Number (Primary)
  • ABMS Membership (Secondary)
  • ABMS Number (Secondary)
  • Flex Field 1
  • Flex Field 2
  • Flex Field 3
  • Flex Field 4
  • Membership ID
  • External ID
  • External Username
  • Program Name
  • Program Title
  • Publications
  • Sites
  • Article Titles
  • Article DOIs
  • Participant Type
  • Registration Date
  • Registration Status
  • Customer Event ID
  • Event Date
  • Venue Name
  • Venue City
  • Venue State
  • Attended Event
  • Fulfillment Qualification
  • Fulfillment Status
  • Manual Fulfillment Date
  • Fulfillment Notes
  • Credit Description
  • Credit Amount
  • Accredited Provider
  • Purchase Date
  • Payment Amount
  • Discount Amount
  • Discount Code(s)
  • Post-Test Score
  • Post-Test Attempts
  • Post-Test Passed
  • Post-Test Date
  • All learner survey answers
  • All learner test answers
  • Total Program View Time


The Complete Program Results export (pictured above) will contain all data for the activity.  The export files available for the individual tests and surveys (see image below) will contain data only for that particular test, survey, or report, including learners’ profile information.

rfc-reports3.jpg


(Back to Table of Contents)

Live Event Reports

In addition to the standard set of reports, live event activities will include additional reports, depending upon whether the activity included registration.  Activities that include event registration will contain a Registration Date and Location report.  If a request for credit is part of the live event activity, a session attendance report and speaker/evaluation reports will also be available.
live-events.jpg

Registration Date and Location

The Registration Date and Location report displays the registration totals for each venue in the activity.  A color-coded chart provides statistics for learners who are marked as registered, promoted from a wait list, waiting list, and walk-ins.
live-events2.jpg


(Back to Table of Contents)
 

Participant Type

The Participant Type report will display a breakdown of registration totals by venue for each participant type.  For this report to display, participant types must be configured for the activity.
live-events3.jpg


(Back to Table of Contents)

Session Attendance Report and Session Details Export

The Session Attendance Report displays learner attendance totals for each session in the activity, along with the total number of registrants for the event and the average session attendance.  If the activity contains multiple venues, attendance totals by session will be provided for each venue. 

The Export Session Attendance Details links (shown in the image above) allow you to download a spreadsheet file containing all the detailed session information. A full list of all the data included in the spreadsheet in listed below:
live-events1.jpg


(Back to Table of Contents)
 

Session and Speaker Evaluation

live-events4.jpg

The Session Evaluation reports allow you to view evaluation data for a single session, all sessions in aggregate (default view), or a group of individually selected sessions.  When opening the report, the default view displays data for all sessions.  To view individual session data, select a session from the list, then click the Update Report button.  To view data for multiple sessions, hold the CTRL button and click multiple sessions, then click the Update Report button.  To switch back to the default aggregate view for all sessions/speakers, click the Reset button. 
live-events5.jpg

The Speaker Evaluation reports will show evaluation data for all speakers in aggregate (default view), all speakers in a single session (if there was more than one), an individual speaker from a session, or all sessions that a speaker presented at.

When opening the report, the default view displays data for all speakers across all sessions.  To view individual speaker data, select the speaker from the list, then click the Update Report button.  To view data for multiple sessions or speakers, hold the CTRL button and click multiple sessions or speakers, then click the Update Report button.  To switch back to the default aggregate view for all sessions/speakers, click the Reset button.
live-events6.jpg


(Back to Table of Contents)

Aggregate Reporting

To access aggregate reporting, make sure you have selected All Programs from the Manage Your Program menu, then click the Reports tab. The Aggregate Reports are compiled using data from all the activities listed in your account (everything contained in the Manage your Program drop-down menu).
aggregate_reports1.jpg


(Back to Table of Contents)

Shared Survey and Test Reports

A set of reports is available for any tests or surveys that are shared across multiple activities. For example, in many cases, the Activity Evaluation for a live or enduring template is a shared/common survey.  Any evaluations that are "built in" to the template are shared/common evaluations. If an evaluation is "built in" to the template, you can only select to include it in an activity but can't edit it.

To access a shared test or survey report, select the survey or test from the drop-down menu, then click the report you want to view.  The numbers in parenthesis indicate the number of activities the test or survey is shared across.

You can also view ALL "unshared" post-tests and surveys for ALL activities by unchecking “Show only shared surveys/tests”  

 


(Back to Table of Contents)

Participation Metrics

Many of the same participation metrics reports that are available for individual activities are also available in aggregate reporting. aggregate_reports2.jpg


(Back to Table of Contents)

Programs Report

From the Program Report tab, the Participation by Program Report displays a high-level summary of activity participation across all activities. This is a great report to quickly find out which activities have the highest number of completed, in-progress, outcomes and total records. Click the column headers to sort the table. You can also use the search box to quickly locate an activity or filter the results. 

aggregate_reports9.jpgaggregate_reports10.jpg 


(Back to Table of Contents)

Credit Report

The All Programs Credit Report displays an interactive bar graph showing data for the type and amount of credit selected across all activities. Hover the cursor over the chart to see the specific counts for each credit amount. You can also click the name of the credit type in the table to see the detailed list of specific learners who selected that credit type. 

aggregate_reports8.jpg


(Back to Table of Contents)

eCommerce Reports

aggregate_reports5.jpg

The Program Sales report provides detailed financial data for all of your eCommerce enabled activities.  For each activity, totals are given for:

  • Quantity
  • Program Sales ($)
  • Program Discounts ($)
  • Quantity (Packaged)
  • Packaged Discounts ($)
  • Packaged Sales ($)
  • Token Sales
  • Token Sales ($)
  • Quantity (net)
  • Net Discount ($)
  • Net Sales ($

Click the column header to sort the table.  You can also use the calendar widget to adjust the time-frame of the report. A link to export the data to Excel is located in the upper right-hand corner. 

aggregate_reports6.jpg

 

The Sales by Discount Code report provides detailed information about the number of times a specific discount code has been used. Use the date filter to view data for a specified time period. 

Individual learner data associated with each discount code is available in the Program Participation Details export.  

aggregate_reports7.jpg


(Back to Table of Contents)

ACCME PARS Reports

From the Download tab, create an ACCME PARS export that includes data from ALL activities in your account (everything listed in the Manage Your Program drop-down menu).

Please refer to the ACCME PARS How-to Guide for more detailed instructions.

aggregate_reports3-2.jpg


(Back to Table of Contents)

Download tab

From the Download tab, there are three different aggregate data files available for download.

aggregate_reports4.jpg

 

The Program Participation Details report contains ALL learner participation records across all activities, including profile, credit, eCommerce, and post-test information. The complete list of information included in the spreadsheet is shown below:

  • Participation ID
  • User ID
  • Code
  • Prefix
  • FName
  • MI
  • LName
  • Gender
  • Birth Date
  • Email
  • Company
  • Job Title
  • Address Line 1
  • Address Line 2
  • City
  • State
  • Zip
  • Country
  • Phone Area Code
  • Phone Number
  • Fax Area Code
  • Fax Number
  • Custom1
  • Custom2
  • Notes
  • Award Date
  • Complete Date
  • Last View Date
  • Expiration Date
  • Recruitment Campaign Id
  • Recruitment Campaign
  • Profession
  • Specialty
  • Specialty2
  • Degree
  • Degree2
  • Designation
  • Client Group
  • Origin URL
  • Site code
  • NPI Number
  • ABMS Membership (Primary)
  • ABMS Number (Primary)
  • ABMS Membership (Secondary)
  • ABMS Number (Secondary)
  • Flex Field 1
  • Flex Field 2
  • Flex Field 3
  • Flex Field 4
  • Membership ID
  • External ID
  • External Username
  • Program Name
  • Program Title
  • Publications
  • Sites
  • Article Titles
  • Article DOIs
  • Participant Type
  • Registration Date
  • Registration Status
  • Customer Event ID
  • Event Date
  • Venue Name
  • Venue City
  • Venue State
  • Attended Event
  • Fulfillment Qualification
  • Fulfillment Status
  • Manual Fulfillment Date
  • Fulfillment Notes
  • Credit Description
  • Credit Amount
  • Accredited Provider
  • Purchase Date
  • Payment Amount
  • Discount Amount
  • Discount Code(s)
  • Post-Test Score
  • Post-Test Attempts
  • Post-Test Passed
  • Post-Test Date
  • Total Program View Time

 

The Program Activity Summary report contains data related to activity information, such as participation totals, outcomes totals, activity start/end dates, and activity type. The complete list of information included in the spreadsheet is shown below:

  • Meeting ID
  • Program Title
  • Program Full Title
  • Complete
  • In Progress
  • Outcomes
  • Unassigned
  • Not Started
  • Total Learners
  • Program Sessions
  • Sessions Attended
  • Anonymous Sessions Attended
  • Start Date
  • End Date
  • Completed
  • Has Outcomes
  • Activity Type
  • Publication
  • Cover Date
  • Cover Date Text
  • Article Titles
  • Article DOIs
  • Site

 

The Live Event Session Participation Details report includes detailed session participation information for all live events. In the new report, you'll see session details such as session name, start and end dates/times, credit, speakers, venues, sites, attendance totals, and more. The complete list of information included in the spreadsheet is shown below:

  • Activity Id
  • Activity Id
  • Activity External Id
  • Activity Name
  • Activity Title
  • Activity Start Date
  • Activity End Date
  • Site
  • Venue
  • City
  • State
  • Session Id
  • Session Name
  • Session Start Time
  • Session Start Time
  • Session End Time
  • Session Credits
  • Session Contributors
  • Session Topics
  • Total Attendance Counts
  • Physicians Attendance Counts
  • Non-Physicians Attendance Counts
  • Anon Physicians Attendance Counts
  • Anon Nonphysicians Attendance Counts
  • Anon Residents Attendance Counts

 

(Back to Table of Contents)