Profile applicability: Level 1
Do not disable timeouts on streaming connections.
Setting idle timeouts ensures that you are protected against Denial-of-Service attacks,
inactive connections and running out of ephemeral ports.
By default,
--streaming-connection-idle-timeout is set to 4 hours which might be too high for your environment. Setting this as appropriate
would additionally ensure that such streaming connections are timed out after serving
legitimate use cases.
NoteSee the Azure AKS documentation for the default value.
|
Impact
Long-lived connections could be interrupted.
Audit
Audit Method 1-
SSH to the relevant node.
-
Run the following command on each node to find the running kubelet process:
ps -ef | grep kubelet
If the command line for the process includes the argumentstreaming-connection-idle-timeout, verify that it is not set to0. -
If the
streaming-connection-idle-timeoutargument is not present, refer to the config argument that specifies the location of the Kubelet config file. Open the Kubelet config file:cat /etc/kubernetes/kubelet/kubelet-config.json
Verify that thestreamingConnectionIdleTimeoutargument is not set to0.
If using the api configz endpoint, search for the status of
"streamingConnectionIdleTimeout":"4h0m0s" by extracting the live configuration from the nodes running kubelet.-
Set the local proxy port and the following variables:
HOSTNAME_PORT="localhost-and-port-number" NODE_NAME="The- Name-Of-Node-To-Extract-Configuration" from the output of "kubectl get nodes"kubectl proxy --port=8001 & export HOSTNAME_PORT=localhost:8001 export NODE_NAME=ip-192.168.31.226.aks.internal curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz"
Remediation
Remediation Method 1If modifying the Kubelet config file, edit the
kubelet-config.json file /etc/kubernetes/kubelet/kubelet-config.json and set the below parameter to a non-zero value in the format of #h#m#s:"streamingConnectionIdleTimeout": "4h0m0s"
Ensure that the kubelet service file
Remediation Method 2
/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf does not specify a --streaming-connection-idle-timeout argument because it would override the Kubelet config file.If using executable arguments, edit the kubelet service file
/etc/systemd/system/kubelet.service.d/10-kubelet-args.conf on each worker node and add the below parameter at the end of the KUBELET_ARGS variable
string:--streaming-connection-idle-timeout=4h0m0sRemediation Method 3
If using the api configz endpoint, search for the status of
"streamingConnectionIdleTimeout": by extracting the live configuration from the nodes running kubelet. See detailed step-by-step configmap procedures in Reconfigure a Node's Kubelet in a Live Cluster, and then rerun the curl statement from audit process to check for kubelet configuration
changes:
kubectl proxy --port=8001 &
export HOSTNAME_PORT=localhost:8001
export NODE_NAME=ip-192.168.31.226.aks.internal
curl -sSL "http://${HOSTNAME_PORT}/api/v1/nodes/${NODE_NAME}/proxy/configz"
For all three remediations:
Based on your system, restart the kubelet service and check status:
systemctl daemon-reload systemctl restart kubelet.service systemctl status kubelet -l
