In this article series, we have been discussing the Understanding OPNFV book (see links to previous articles below). Here, we will look at OPNFV continuous testing and writing and onboarding virtual network functions (VNFs).
As we have discussed previously, OPNFV integrates a number of network function virtualization (NFV) related upstream projects and continuously tests specific combinations. The test projects in OPNFV are meant to validate the numerous scenarios using multiple test cases. The following figure shows a high-level view of the OPNFV testing framework.
OPNFV testing projects may be viewed through four distinct lenses: coverage, scope, tier, and purpose. Coverage determines whether a test covers the entire stack, a specific subsystem (e.g., OpenStack) or just one component (e.g., OVS) of an OPNFV scenario, which is a specific integration of upstream projects — a reference architecture. Scope is classified into functional, performance, stress and compliance testing. A tier, on the other hand, describes the complexity (end-to-end network service vs. smoke) and frequency (daily vs. weekly) of a particular test. Finally, the purpose defines why a test is included (e.g., to gate a commit or or to simply be informational).
The following three broad OPNFV testing projects and five sub-projects are covered in the book:
-
Functest: Provides functional testing and validation of various OPNFV scenarios. Functest includes subsystem and end-to-end tests (e.g., Clearwater vIMS).
-
Yardstick: Executes performance testing on OPNFV scenarios based on ETSI reference test suites. The five sub-projects are as follows:
-
VSPERF: Virtual switch (OVS, FD.io) performance testing
-
Cperf: SDN controller control-plane and dataplane performance testing
-
Storperf: External block storage performance testing
-
QTIP: Compute benchmarking (Note: The Euphrates release includes storage benchmarking as well)
-
Bottlenecks: Stress testing
-
-
Dovetail: Includes compliance tests. Dovetail will form the foundation of the future OPNFV Compliance Verification Program (CVP) for NFV infrastructure, VIM, and SDN controller commercial products.
Note that since publication of the book, two more testing-related projects have been included in the Euphrates release — Sample VNF and NFVBench.
The QTIP project is described in the book as follows:
Remember benchmarks such as MIPS or TPC-C, which attempted to provide a measure of infrastructure performance through one single number? QTIP attempts to do the same for NVFI compute (storage and networking part of roadmap) performance. QTIP is a Yardstick plugin that collects metrics from a number of tests selected from five different categories: integer, floating point, memory, deep packet inspection and cipher speeds. These numbers are crunched to produce a single QTIP benchmark. The baseline is 2,500, and bigger is better! In that sense one of the goal of QTIP is to make Yardstick results very easy to consume.
Another attractive feature of OPNFV is being able to view tests results in real-time through a unified dashboard. Behind the scenes, the OPNFV community has made a significant investment in test APIs, testcase database, results database, and results visualization efforts. Scenario reporting results are available for Functest, Yardstick, Storperf, and VSPERF.
The entire OPNFV stack, ultimately, serves one purpose: to run virtual network functions that in turn constitute network services. Chapter 9, which is meant for VNF vendors, looks at two major considerations: how to write VNFs and how to onboard them. After a brief discussion of modeling languages and general principles around VNF packaging, the book gives a concrete example where the Clearwater virtual IP multimedia system (vIMS) VNF is onboarded and tested on OPNFV along with the Cloudify management and orchestration software.
The following figure shows how Clearwater vIMS — a complex cloud native application with a number of interconnected virtual instances — is initially deployed on OPNFV. Once onboarded, end-to-end tests in Functest fully validate the functionality of the VNF along with the underlying VIM, NFVI, and SDN controller functionality.
Want to learn more? You can download the Understanding OPNFV ebook in PDF (in English or Chinese), or order a printed version on Amazon. Or you can check out the previous blogs:
Also, check out the next OPNFV Plugfest, where members and non-members come together to test OPNFV along with other open source and commercial products.
