Ndt (network diagnostic tool) – m-lab

Contents

• Run an NDT Test
• History
• Run an NDT Test
• History
• NDT Testing Protocols
• Source code
• Citing the M-Lab NDT Dataset
• Data Collected by NDT
• NDT Data in Raw Format
• NDT Data in BigQuery
• Current BigQuery Tables/Views
• Planned BigQuery Tables/Views (Publish M-Lab 2. Global Pilot)
  • I’m as mad as hell, and I’m not going to take this anymore! Speech from Network (1080p)

NDT is really a single stream performance measurement of the connection’s convenience of “bulk transport” (as defined in IETF’s RFC 3148. NDT measures “single stream performance” or “bulk transport capacity”. NDT reports upload and download speeds and latency metrics, and attempts to determine which problems limit speeds.

Run an NDT Test

If you are looking at running an NDT test, check out our standalone speed test site: https://speed.measurementlab.internet

History

NDT is really a single stream performance measurement of the connection’s convenience of “bulk transport” (as defined in IETF’s RFC 3148. NDT measures “single stream performance” or “bulk transport capacity”. NDT reports upload and download speeds and latency metrics, and attempts to determine which problems limit speeds.

Run an NDT Test

If you are looking at running an NDT test, check out our standalone speed test site: https://speed.measurementlab.internet

History

From 2009-2019, M-Lab has run the default NDT server initially produced by Internet2. This version uses the web100 linux kernel extension for gathering data points concerning the TCP connection.

Through the finish of 2019, M-Lab will launch a totally re-written ndt-server, supplying the ndt5 and ndt7 protocols. M-Lab will also transition located experiments to make use of the netlink socket and TCP_INFO kernel instrumentation in 2019, replacing our reliance upon Web100. The web100 form of server is going to be decommissioned on M-Lab once ndt-server continues to be tested and launched. M-Lab will retain data collected although it is at production will come in the web100 tables referenced below.

NDT Testing Protocols

As part of our transition in the web100 form of NDT server towards the new platform, M-Lab has named specific protocol versions for that current server and also the brand new one we’re testing.

• web100 may be the protocol refering to data collected through the current NDT server
• ndt5 is really a new NDT protocol made to be backward suitable for current clients
• ndt7 is really a new NDT protocol that operates exclusively over TLS port 443, and can provide new measurement methods and abilities

Source code

• web100 historic ndt
• ndt-server

Citing the M-Lab NDT Dataset

Please cite the NDT data set the following: The M-Lab NDT Data Set, https://measurementlab.internet/tests/ndt

or, in BibTeX format:

@misc”,

title=”The -ab Data Set”,

year=”(2009-02-11 — 2015-12-21)”,

howpublished=”urlinternet/tests/ndt”,

comment=”Based on should you used viz.measurementlab.internet, bigquery, or even the raw data, please use among the following notes:”,

note=”Bigquery table “,

note1=”Google cloud storage “,

note2=”Data visualization system urlinternet”,

Data Collected by NDT

Advertising media are NDT, the Ip supplied by your Isp is going to be collected together with your measurement results. M-Lab conducts the exam and publishes all test leads to promote Research. NDT doesn’t collect any details about you being an Web surfer.

Please review M-Lab’s Online Privacy Policy to understand information is collected and just how information is used before initiating an evaluation.

NDT Data in Raw Format

Data collected by NDT will come in raw format in the search engines Cloud Storage: https://console.cloud.google.com/storage/browser/archive-measurement-lab/ndt.

Advanced users can also be thinking about acquiring raw M-Lab test data for detailed analyses. For instance, TCP packet captures are conducted for every NDT test, and therefore are only accessible in M-Lab’s raw data archives. Information on how M-Lab publishes test data in raw form is supplied on the Google Cloud Storage documentation page.

NDT Data in BigQuery

To create NDT data more designed for research and analysis, M-Lab parses all NDT data into BigQuery tables and views, and makes query access readily available for free by subscription to some Google Group. Learn more about ways to get access on the BigQuery QuickStart page.

M-Lab BigQuery tables/views for NDT data have been in transition by Sept. 2019, because the team completes our Global Pilot from the M-Lab 2. platform. Here we provide two lists of BigQuery tables/views: our current tables and views, and individuals which M-Lab will transition to through the finish from the global pilot.

Current BigQuery Tables/Views

• Useful tables/views – some tables/views which are pre-filtered for generally used queries, and come from the “faithful” tables/views
  • measurement-lab.ndt.downloads
    • A subset look at all NDT download tests from measurement-lab.ndt.suggested where:
      • A minimum of 8 KB of information was transferred
      • Test duration was between 9 and a minute
      • Congestion was detected
  • measurement-lab.ndt.uploads
    • A subset look at all NDT upload tests from measurement-lab.ndt.suggested in which a sensible final amount of bytes was received (8192)
  • measurement-lab.ndt.suggested
    • A subset look at all NDT upload and download tests from measurement-lab.ndt.web100 where:
      • TCP finish condition is smart
      • Test duration was between 9 and a minute
    • except:
      • Internal M-Lab finish-to-finish monitoring tests
      • Tests marked as blacklisted

Planned BigQuery Tables/Views (Publish M-Lab 2. Global Pilot)

• Faithful tables/views – the bottom tables/views for every NDT data type
  • measurement-lab.ndt.web100
    • NDT data collected while using web100 protocol, while using web100 Linux kernel patch for those TCP metrics.
    • web100 is root BigQuery view that all current “Helpful” views are derived.
    • web100 description and schema
  • measurement-lab.ndt.ndt5
    • NDT data collected while using ndt5 protocol on or after 2019-07-19, using tcp-info for those TCP metrics.
    • ndt5 description and schema
  • ndt7 (not far off)
    • NDT data collected while using ndt7 protocol using tcp-info for those TCP metrics.
    • ndt7 description and schema

Resourse: https://measurementlab.internet/tests/ndt/

I’m as mad as hell, and I’m not going to take this anymore! Speech from Network (1080p)