ENVI: Elastic resource flexing for Network function VIrtualization

Sample features for Suricata and Squid


Abstract

Resource flexing is the notion of allocating resources on-demand as workload changes. This is a key advantage of Virtualized Network Functions (VNFs) over their non-virtualized counterparts. It is difficult to balance the timeliness and resource efficiency when making resource flexing decisions due to unpredictable workloads and complex VNF processing logic.
In this work, we propose an Elastic resource flexing system for Network functions VIrtualization (ENVI) that leverages a combination of VNF-level features and infrastructure-level features to construct a neural-network-based scaling decision engine for generating timely scaling decisions. To adapt to dynamic workloads, we design a window-based rewinding mechanism to update the neural network with emerging workload patterns and make accurate decisions in real time. Our experimental results for real VNFs (IDS Suricata and caching proxy Squid) using workloads generated based on real-world traces, show that ENVI provisions significantly fewer (up to 26%) resources without violating service level objectives, compared to commonly used rule-based scaling policies.

Suricata sample features

i
vnf.Available_number_of_file_descriptorsvnf.Store_Disk_files_openvnf.page_faults
vnf.Average_HTTP_requests_per_minute_since_startvnf.Total_accountedvnf.select_fds
vnf.Average_ICP_messages_per_minute_since_startvnf.Total_freevnf.select_loops
vnf.CPU_Timevnf.Total_in_usevnf.server.all.errors
vnf.Cache_Hitsvnf.Total_sizevnf.server.all.kbytes_in
vnf.Cache_Missesvnf.Total_space_in_arenavnf.server.all.kbytes_out
vnf.DNS_Lookupsvnf.UP_Timevnf.server.all.requests
vnf.Disk_hits_as_%_of_hit_requestsvnf.aborted_requestsvnf.server.ftp.errors
vnf.Files_queued_for_openvnf.average_select_fd_periodvnf.server.ftp.kbytes_in
vnf.Free_Ordinary_blocksvnf.client_http.all_median_svc_timevnf.server.ftp.kbytes_out
vnf.Free_Small_blocksvnf.client_http.errorsvnf.server.ftp.requests
vnf.HTTP_Requests_(All)vnf.client_http.hit_median_svc_timevnf.server.http.errors
vnf.Hits_as_%_of_all_requestsvnf.client_http.hitsvnf.server.http.kbytes_in
vnf.Hits_as_%_of_bytes_sentvnf.client_http.kbytes_invnf.server.http.kbytes_out
vnf.Holding_blocksvnf.client_http.kbytes_outvnf.server.http.requests
vnf.ICP_Queriesvnf.client_http.miss_median_svc_timevnf.server.other.errors
vnf.Largest_file_desc_currently_in_usevnf.client_http.nh_median_svc_timevnf.server.other.kbytes_in
vnf.Maximum_Resident_Sizevnf.client_http.nm_median_svc_timevnf.server.other.kbytes_out
vnf.Maximum_number_of_file_descriptorsvnf.client_http.requestsvnf.server.other.requests
vnf.Mean_Object_Sizevnf.cpu_timevnf.service_time_10sec
vnf.Memory_hits_as_%_of_hit_requestsvnf.cpu_usagevnf.swap.files_cleaned
vnf.Near_Hitsvnf.dns.median_svc_timevnf.swap.ins
vnf.Not-Modified_Repliesvnf.icp.kbytes_recvvnf.swap.outs
vnf.Number_of_HTCP_messages_receivedvnf.icp.kbytes_sentvnf.syscalls.disk.closes
vnf.Number_of_HTCP_messages_sentvnf.icp.pkts_recvvnf.syscalls.disk.opens
vnf.Number_of_HTTP_requests_receivedvnf.icp.pkts_sentvnf.syscalls.disk.reads
vnf.Number_of_ICP_messages_receivedvnf.icp.q_kbytes_recvvnf.syscalls.disk.seeks
vnf.Number_of_ICP_messages_sentvnf.icp.q_kbytes_sentvnf.syscalls.disk.unlinks
vnf.Number_of_clients_accessing_cachevnf.icp.queries_recvvnf.syscalls.disk.writes
vnf.Number_of_file_desc_currently_in_usevnf.icp.queries_sentvnf.syscalls.sock.accepts
vnf.Number_of_queued_ICP_repliesvnf.icp.query_median_svc_timevnf.syscalls.sock.binds
vnf.Ordinary_blocksvnf.icp.query_timeoutsvnf.syscalls.sock.closes
vnf.Page_faults_with_physical_i/ovnf.icp.r_kbytes_recvvnf.syscalls.sock.connects
vnf.Process_Data_Segment_Size_via_sbrk()vnf.icp.r_kbytes_sentvnf.syscalls.sock.reads
vnf.Request_failure_ratiovnf.icp.replies_queuedvnf.syscalls.sock.recvfroms
vnf.Requests_given_to_unlinkdvnf.icp.replies_recvvnf.syscalls.sock.sendtos
vnf.Reserved_number_of_file_descriptorsvnf.icp.replies_sentvnf.syscalls.sock.sockets
vnf.Select_loop_calledvnf.icp.reply_median_svc_timevnf.syscalls.sock.writes
vnf.Small_blocksvnf.median_select_fdsvnf.time_stamp_sec
vnf.Storage_Mem_capacityvnf.memPoolAlloc_callsvnf.time_stamp_sec_delta
vnf.Storage_Mem_sizevnf.memPoolFree_callsvnf.unlink.requests
vnf.Storage_Swap_capacityvnf.memPool_accountedvnf.wall_time
vnf.Storage_Swap_sizevnf.memPool_unaccounted

Suricata sample features

vnf.app_layer.flow.dcerpc_tcpvnf.decoder.max_pkt_sizevnf.flow_mgr.flows_notimeout
vnf.app_layer.flow.dcerpc_udpvnf.decoder.mplsvnf.flow_mgr.flows_removed
vnf.app_layer.flow.dns_tcpvnf.decoder.nullvnf.flow_mgr.flows_timeout
vnf.app_layer.flow.dns_udpvnf.decoder.pktsvnf.flow_mgr.flows_timeout_inuse
vnf.app_layer.flow.failed_tcpvnf.decoder.pppvnf.flow_mgr.new_pruned
vnf.app_layer.flow.failed_udpvnf.decoder.pppoevnf.flow_mgr.rows_busy
vnf.app_layer.flow.ftpvnf.decoder.rawvnf.flow_mgr.rows_checked
vnf.app_layer.flow.httpvnf.decoder.sctpvnf.flow_mgr.rows_empty
vnf.app_layer.flow.imapvnf.decoder.sllvnf.flow_mgr.rows_maxlen
vnf.app_layer.flow.msnvnf.decoder.tcpvnf.flow_mgr.rows_skipped
vnf.app_layer.flow.smbvnf.decoder.teredovnf.http.memcap
vnf.app_layer.flow.smtpvnf.decoder.udpvnf.http.memuse
vnf.app_layer.flow.sshvnf.decoder.vlanvnf.ips.accepted
vnf.app_layer.flow.tlsvnf.decoder.vlan_qinqvnf.ips.blocked
vnf.app_layer.tx.dns_tcpvnf.defrag.ipv4.fragmentsvnf.ips.rejected
vnf.app_layer.tx.dns_udpvnf.defrag.ipv4.reassembledvnf.ips.replaced
vnf.app_layer.tx.httpvnf.defrag.ipv4.timeoutsvnf.tcp.invalid_checksum
vnf.app_layer.tx.smtpvnf.defrag.ipv6.fragmentsvnf.tcp.memuse
vnf.app_layer.tx.tlsvnf.defrag.ipv6.reassembledvnf.tcp.no_flow
vnf.decoder.avg_pkt_sizevnf.defrag.ipv6.timeoutsvnf.tcp.pseudo
vnf.decoder.bytesvnf.defrag.max_frag_hitsvnf.tcp.pseudo_failed
vnf.decoder.dce.pkt_too_smallvnf.detect.alertvnf.tcp.reassembly_gap
vnf.decoder.erspanvnf.dns.memcap_globalvnf.tcp.reassembly_memuse
vnf.decoder.ethernetvnf.dns.memcap_statevnf.tcp.rst
vnf.decoder.grevnf.dns.memusevnf.tcp.segment_memcap_drop
vnf.decoder.icmpv4vnf.flow.emerg_mode_enteredvnf.tcp.sessions
vnf.decoder.icmpv6vnf.flow.emerg_mode_overvnf.tcp.ssn_memcap_drop
vnf.decoder.invalidvnf.flow.memcapvnf.tcp.stream_depth_reached
vnf.decoder.ipraw.invalid_ip_versionvnf.flow.memusevnf.tcp.syn
vnf.decoder.ipv4vnf.flow.sparevnf.tcp.synack
vnf.decoder.ipv4_in_ipv6vnf.flow.tcp_reusevnf.time_stamp_sec
vnf.decoder.ipv6vnf.flow_mgr.bypassed_prunedvnf.time_stamp_sec_delta
vnf.decoder.ipv6_in_ipv6vnf.flow_mgr.closed_prunedvnf.uptime
vnf.decoder.ltnull.pkt_too_smallvnf.flow_mgr.est_pruned
vnf.decoder.ltnull.unsupported_typevnf.flow_mgr.flows_checked