Description of Methodology (DOM)
© 2017 Hulu, LLC | all rights reserved
Hulu, LLC | 2500 Broadway, 2nd Floor | Santa Monica, CA 90404 | (310) 571-4700 | hulu.com
1.2 | last updated July 19, 2017
Frequency of Updates
Owner of Update Process
No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, translated into any other language in any form or by any means, electronic or mechanical, including photocopying or recording, for any purpose, without the express permission of Hulu, LLC.
This document is provided for informational purposes. The information is provided “as is” and without warranties of any kind. Due to continuing product development, the information is subject to change without notice. Hulu, LLC shall not be responsible or liable for any inaccuracies, errors, or outdated information.
This document is intended for advertisers. It provides a summary of the video-streaming measurement processes employed by Hulu, LLC (Hulu). It includes a general description of Hulu’s measurement methodology, filtration processes and reporting procedures. Issuance of this document is in accordance with Hulu’s accreditation conferred by the Media Rating Council (MRC).
Hulu provides an Ad-supported streaming digital video service that offers a selection of TV shows, clips and movies via its video players at hulu.com, embedded Hulu video players hosted at other internet sites, mobile devices via the Hulu mobile application and over-the-top (OTT) internet-connected devices via device-specific Hulu applications. Users of Hulu’s digital video service may elect to view content on hulu.com and through external sites such as yahoo.com. To view Hulu content on web, or any Mobile or OTT internet-connected device, Hulu users must subscribe to the Hulu service.
Video Ads are served within the Hulu player at pre- and mid-roll positions. Companion display Ads may also be shown within these placements, or adjacent to the player, in close proximity.
Hulu partners with Content Providers (CPs) and distributors to provide digital video content to users. Hulu also produces certain original video content available through its streaming video service. Regardless of the origin of the content, Hulu maintains control of its streaming digital video players and content pipelines, and encodes all video content and advertisements prior to distribution within Hulu’s proprietary video players on Web and within Hulu applications.
Access to Hulu content is limited to users within the United States, including US territories and military bases overseas, based on IP-address geolocation as determined by a third-party vendor; other regions are blocked.
In most situations, Hulu’s first-party Ad server performs Ad decisioning. However, Hulu also supports third-party Ad serving in certain situations involving pre-established relationships with specified content providers.
The following Hulu metrics are included in MRC-accreditation scope, on the Web, Mobile Web, and Mobile In-App platforms:
- Impressions, which are fired by Hulu clients once a video or rich-media Ad begins playing, or once a display Ad renders.
- Completions, which are fired after a video or rich-media Ad finishes playing.
- Their derivatives – Completion Rate and Delivered Percent.
The following Hulu reports are included in MRC-accreditation scope:
Excluded from Scope
The following items are excluded from MRC-accreditation scope:
- All Hulu metrics not mentioned above in Accreditation Scope.
- All Hulu reports not mentioned above in Accreditation Scope.
- All metrics on the OTT platform, including Impressions and Completions. As of the time of this writing, MRC-accreditation requirements for OTT have not been published.
- Akamai, a content-distribution network used by Hulu.
- Third-party Ad servers that are sometimes involved in Hulu’s Ad serving are excluded. These third-party Ad servers may possess their own MRC accreditations.
- Third-party metrics, including those that measure Hulu-served Ads.
Client-Initiated Beacons Only
All Hulu beacons originate directly from the client. No server-side beacons are used. No proxies are intentionally used.
Hulu does not use sampling. Measurements are counted directly from client-initiated beacons, after filtration processes mentioned below.
Content-Distribution Network (CDN) Usage
Clients send Impressions and Completions to geographically-dispersed CDNs provided by Akamai. Akamai’s CDNs then forward these beacons to Hulu’s measurement servers.
Client initiate Impressions and Completions by firing a beacon via HTTP GET request.
The measurement server returns a 200 to indicate successful reception of the beacon, and writes a line in its log file to record the Impression or Completion.
The above is true of Impressions and Completions for all video, rich-media, and display Ads.
Network latency may result in beacons not arriving at the server around the moment an Impression or Completion occurs, but rather at some point afterward. If this is case, the Impression or Completion is counted upon receipt.
Network issues may also cause beacons being duplicated somewhere along the way (e.g., by a third-party proxy server) between Hulu’s client and server. Hulu addresses this risk in two ways. First, by never performing beacon retries on any accreditation platform (i.e., Web, Mobile Web, or Mobile In-App). Second, by performing beacon de-duplication on the server side.
No Compound Tracking
Hulu does not compound or grouped beacons. Measurements are counted directly from client-initiated beacons one at a time.
Hulu uses a combination of beacon parameters, including a client-generated auto-incrementing “seq” (i.e., sequence) value, to create uniqueness of Impressions and Completions.
Also, in the 200 response to Impression and Completion beacons, the server uses the following Header Controls:
Cache-Control: max-age=0, no-cache, no-store
These values instruct the client to not cache the GET request, and always fire a new instance of the beacon.
Hulu does not engage in auto-play of Video or Interactive Ads. Playback is never initiated without an explicit user action (i.e., click or tap).
Hulu engages in continuous play, which is automatic advancement to a new piece of Content after an existing piece of Content comes to an end.
Hulu uses a threshold to determine when to show an “Are you still watching?” confirmation. After four hours of continuous play without user interaction, the dialog is presented, and if no confirmation (i.e., ‘Yes’) is received from the viewer, further playback is stopped.
Based on a preliminary study, ~18% of Impressions on Web was generated by sessions that included 1+ hour inactivity. ~1% of Impressions on Mobile In-App was generated by sessions that included 1+ hour of inactivity. Hulu is developing and conducting improved studies on viewer inactivity in order to update its continuous-play functionality.
Impression and Completion beacons received from Akamai are immediately written into log files. The log files contain raw beacon data. They undergo processing before they can be consumed by reporting mechanisms.
Daily processing allows Hulu’s reports to include the previous day’s data, but not the current day’s.
Ad-blocking software is capable of preventing Hulu’s Ads from being displayed.
When these situations occur, because no Ads are loaded, no Impression or Completions are fired.
Hulu employs a suite of techniques in an attempt to identify and filter invalid activity. These techniques include but are not limited to known and suspected non-human activity and suspected invalid human activity.
Because user identification and intent cannot always be known or discerned by the publisher, advertiser or their respective agents, it is unlikely that all invalid activity can be identified and excluded from report results.
Below, Hulu’s various techniques are summarized strictly at a high-level for security purposes.
General Invalid-Traffic Filtration (GIVT)
Hulu practices GIVT.
Sophisticated Invalid-Traffic Filtration (SIVT) is not performed.
Hulu performs its own filtration without the assistance of any third parties.
Hulu removes all internally-generated traffic.
IAB Blacklist and Whitelist
Based on User Agent, Hulu performs the IAB dual-pass method to remove all traffic that doesn’t appear on the international lists of spiders and bots, and valid browsers.
Activity-Based Filtration (ABF)
Hulu employs ABF to identify anomalies. The specifics of its ABF techniques are kept confidential for security purposes. They are disclosed only to MRC auditors as part of the audit process.
Hulu does not employ any pre-fetching mechanisms. Impression and Completion beacons are fired in real time.
Filtration is applied against Hulu’s entire traffic across the Web, Mobile Web, and Mobile In-App platforms. No subsets or sampling techniques are used.
When Impression and Completion beacons are received by the server, they undergo validation to ensure critical parameters aren’t missing. They are also evaluated to see whether parameter values are contradictory. Any beacons that fail either type of validation are discarded ahead of any further processing.
In the unlikely event Hulu’s log files are corrupted, Akamai is capable of resending beacons for Hulu to reprocess.
An analysis of IVT during Q4 of 2016 shows that IVT typically fluctuates at immaterial levels.
When significant changes to Hulu’s filtration methods and / or technology are implemented, Hulu issues a statement to clients with active or upcoming Ad Campaigns. This is reserved for situations where notification provides a discernible benefit to clients. For security purposes, Hulu discloses as little as possible when it comes to its filtration techniques. Disclosures will be accordingly high-level.
Hulu reports a variety of metrics to clients on a per-Campaign basis.
The reports in accreditation scope – Post-Buy and Device – are presented to clients at multiple points in the Campaign, depending on Campaign length and client preference. The Post-Buy is always presented after a Campaign ends.
Estimation is not used. Report metrics are composed solely from actual data collected in the measurement process.
While report data accumulates on a day-by-day basis, it is considered final as, once processed, a day’s data is official and generally unchanging. Data revisions are possible but rare.
These are Impressions and Completions, and their derivatives – Completion Rate and Delivered Percent.
Impressions and Completions are segregated in three ways:
- Platform. This breaks down into Web, Mobile Web, Mobile In-App, and Living Room. The breakdown allows clients to quickly identify MRC-accredited numbers (i.e., Web, Mobile Web, and Mobile In-App).
- Position. This breaks down into pre-roll, mid-roll, and post-roll.
- Gross (i.e., pre-filtration) vs. net (i.e., post-filtration) numbers.]
Non-accredited metrics are also presented. These include, but are not limited to: quartiles, Clicks, Clickable Impressions and Completions, Click-Through Rate (CTR), Time Spent Viewing, and Unique Users.
Open columns with no metrics or data can also be presented. They are meant to reserve space conveniently for manual data entry. These include Estimated Spend values, Third Party and Discrepancy.
Reports can be:
- Run against a specified date range. Typically the entire duration of the Campaign is used.
- Grouped by day, week, or month. Reports use the time zone Eastern Time (ET). A day is defined as the 24-hour period that begins at 12:00 a.m. and ends at 11:59 p.m. ET. A week is defined as the seven-day period that could span Sunday to Saturday, or Monday to Sunday, depending on client preference. Specific dates are provided in reporting for reference.
- Broken down at the Creative (i.e., individual instances of an Ad type) level, or grouped at the Placement (i.e., Ad type) level instead.
Campaigns parameters are entered by Hulu’s Campaign Coordinators. Before being set live, every Campaign is subject to full review by a Campaign Manager.
After going live, Campaigns undergo pacing monitoring by Campaign Managers to ensure they’re delivering as expected. Optimizations and debugging are performed if necessary.
Revised metrics, including Impressions and Completions, will be communicated when excluded or invalid activity is detected for a Campaign after initial reporting and within three days after the end of the Campaign. The methods of communication include e-mail statement and / or updated reporting.
IVT Discrepancy Resolution
Campaigns billed against Hulu metrics are never billed for IVT traffic. In other words, Ad Campaign targets are always met after discounting any traffic identified as IVT.
However, as mentioned above, clients are always notified of gross vs. net numbers. After reviewing these, clients may contact their Hulu Campaign Manager to discuss any general questions around IVT volume and classification.
Source data for Hulu’s reports is retained indefinitely without expiry.
Records of data reissuance are also retained indefinitely.
At the time of this writing, Hulu maintains a relationship with many major business partners.
The partners that have an influence on Hulu Ad delivery, and carry a material impact based on volume, are:
Other relationships exist but are not listed due to not meeting the criteria mentioned above.
In order to ensure it’s working with reputable partners, Hulu follows a qualification process which includes ascertaining the partner’s stance and progress on anti-IVT measures. Qualification is more stringent for less-known partners, and more relaxed for “household names” and partners with whom Hulu has had long-term cooperation.