On relay search we show an amber dot next to the relay nickname when it is overloaded.
중계기가 과부하됐다는 건 한 개 이상의 부하 메트릭스에 트리거가 걸렸다는 걸 뜻해요.
중계기의 과부하가 발생하고 복구된 후, 약 72시간 동안 Tor측에서 중계기가 과부하에 걸렸음을 안내해요.
중계기가 과부하되었다면 다음과 같이 하세요:
https://status.torproject.org/ 에 들어간 후, "Tor network" 카테고리 내에 보고된 이슈가 있었는지 점검하세요.
네트워크, 메모리, CPU 부하 측면에서 시스템의 sysctl을 조정해보는 걸고려해보세요.
상황을 잘 파악할 수 있도록 MetricsPort를 활성화하는 걸 고려해보세요.
네트워크, 메모리, CPU 부하 측면에서 sysctl 의 조정
TCP 포트 고갈
TCP 포트 고갈 문제를 겪고 있다면, 로컬 포트 범위를 확장하는 걸 고려하세요.
다음을 통해 로컬 포트 범위를 확장할 수 있어요:
# sysctl -w net.ipv4.ip_local_port_range="15000 64000"
또는
# echo 15000 64000 > /proc/sys/net/ipv4/ip_local_port_range
Keep in mind that tuning sysctl as described is not permanent and will be lost upon restart.
You need to add the configuration to /etc/sysctl.conf or to a file in /etc/sysctl.d/ to make it permanent.
MetricsPort
To understand the well-being of Tor relays and the Tor network it is vital to provide and have access to relay metrics.
Relay overload information has been added to relay descriptors since 0.4.6+ but it was not until Tor >= 0.4.7.1-alpha that an interface to the underlying relay metrics was available: the metrics port.
MetricsPort를 활성화하기
Tor provides access to the metrics port via a torrc configuration option called MetricsPort.
It's important to understand that exposing the tor MetricsPort publicly is dangerous for the Tor network users, which is why that port is not enabled by default and its access has to be governed by an access policy.
이 포트를 열기 전 사전 준비를 철저하게 하세요. 디버깅이 끝나는 대로 바로 포트를 닫으셔야 해요.
Let's assume you are the only user on a server that runs a Tor relay.
You can enable the metrics port adding this to your torrc file:
MetricsPort 127.0.0.1:9035
MetricsPortPolicy accept 127.0.0.1
And then you will be able to easily retrieve the metrics with:
# curl https://127.0.0.1:9035/metrics
초기 설정 상 Prometheus 포맷이 기본으로 설정돼있어요.
Note: every user on that server will be able to access those relay metrics in the example above.
In general, set a very strict access policy with MetricsPortPolicy and consider using your operating systems firewall features for defense in depth.
For a more detailed explanation about MetricsPort and MetricsPortPolicy see tor's man page.
MetricsPort output
Here is an example of what output enabling MetricsPort will produce (we omitted any congestion control related metrics as we still need to stabilize that interface):
# HELP tor_relay_connections Total number of opened connections
# TYPE tor_relay_connections gauge
tor_relay_connections{type="OR listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="OR listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="OR listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="OR listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="OR",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="OR",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="OR",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="OR",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Exit",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Exit",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Exit",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Exit",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Socks listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Socks listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Socks listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Socks listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Socks",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Socks",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Socks",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Socks",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Directory listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Directory listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Directory listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Directory listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Directory",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Directory",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Directory",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Directory",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Control listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Control listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Control listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Control listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Control",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Control",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Control",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Control",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Transparent pf/netfilter listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Transparent pf/netfilter listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Transparent pf/netfilter listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Transparent pf/netfilter listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Transparent natd listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Transparent natd listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Transparent natd listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Transparent natd listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="DNS listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="DNS listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="DNS listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="DNS listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Extended OR",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Extended OR",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Extended OR",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Extended OR",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Extended OR listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Extended OR listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Extended OR listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Extended OR listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="HTTP tunnel listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="HTTP tunnel listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="HTTP tunnel listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="HTTP tunnel listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Metrics listener",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Metrics listener",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Metrics listener",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Metrics listener",direction="received",state="opened",family="ipv6"} 0
tor_relay_connections{type="Metrics",direction="initiated",state="opened",family="ipv4"} 0
tor_relay_connections{type="Metrics",direction="initiated",state="opened",family="ipv6"} 0
tor_relay_connections{type="Metrics",direction="received",state="opened",family="ipv4"} 0
tor_relay_connections{type="Metrics",direction="received",state="opened",family="ipv6"} 0
# HELP tor_relay_connections_total Total number of created/rejected connections
# TYPE tor_relay_connections_total counter
tor_relay_connections_total{type="OR listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="OR listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="OR listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="OR listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="OR listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="OR listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="OR",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="OR",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="OR",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="OR",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="OR",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="OR",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Exit",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Exit",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Exit",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Exit",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Exit",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Exit",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Socks listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Socks listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Socks listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Socks listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Socks listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Socks listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Socks",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Socks",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Socks",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Socks",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Socks",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Socks",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Directory listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Directory listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Directory listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Directory listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Directory listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Directory listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Directory",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Directory",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Directory",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Directory",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Directory",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Directory",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Control listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Control listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Control listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Control listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Control listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Control listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Control",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Control",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Control",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Control",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Control",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Control",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Transparent pf/netfilter listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Transparent pf/netfilter listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Transparent pf/netfilter listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Transparent pf/netfilter listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Transparent pf/netfilter listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Transparent pf/netfilter listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Transparent natd listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Transparent natd listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Transparent natd listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Transparent natd listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Transparent natd listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Transparent natd listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="DNS listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="DNS listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="DNS listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="DNS listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="DNS listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="DNS listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Extended OR",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Extended OR",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Extended OR",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Extended OR",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Extended OR",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Extended OR",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Extended OR listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Extended OR listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Extended OR listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Extended OR listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Extended OR listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Extended OR listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="HTTP tunnel listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="HTTP tunnel listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="HTTP tunnel listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="HTTP tunnel listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="HTTP tunnel listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="HTTP tunnel listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Metrics listener",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Metrics listener",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Metrics listener",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Metrics listener",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Metrics listener",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Metrics listener",direction="received",state="rejected",family="ipv6"} 0
tor_relay_connections_total{type="Metrics",direction="initiated",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Metrics",direction="initiated",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Metrics",direction="received",state="created",family="ipv4"} 0
tor_relay_connections_total{type="Metrics",direction="received",state="created",family="ipv6"} 0
tor_relay_connections_total{type="Metrics",direction="received",state="rejected",family="ipv4"} 0
tor_relay_connections_total{type="Metrics",direction="received",state="rejected",family="ipv6"} 0
# HELP tor_relay_flag Relay flags from consensus
# TYPE tor_relay_flag gauge
tor_relay_flag{type="Fast"} 0
tor_relay_flag{type="Exit"} 0
tor_relay_flag{type="Authority"} 0
tor_relay_flag{type="Stable"} 0
tor_relay_flag{type="HSDir"} 0
tor_relay_flag{type="Running"} 0
tor_relay_flag{type="V2Dir"} 0
tor_relay_flag{type="Sybil"} 0
tor_relay_flag{type="Guard"} 0
# HELP tor_relay_circuits_total Total number of circuits
# TYPE tor_relay_circuits_total gauge
tor_relay_circuits_total{state="opened"} 0
# HELP tor_relay_streams_total Total number of streams
# TYPE tor_relay_streams_total counter
tor_relay_streams_total{type="BEGIN"} 0
tor_relay_streams_total{type="BEGIN_DIR"} 0
tor_relay_streams_total{type="RESOLVE"} 0
# HELP tor_relay_traffic_bytes Traffic related counters
# TYPE tor_relay_traffic_bytes counter
tor_relay_traffic_bytes{direction="read"} 0
tor_relay_traffic_bytes{direction="written"} 0
# HELP tor_relay_dos_total Denial of Service defenses related counters
# TYPE tor_relay_dos_total counter
tor_relay_dos_total{type="circuit_rejected"} 0
tor_relay_dos_total{type="circuit_killed_max_cell"} 0
tor_relay_dos_total{type="circuit_killed_max_cell_outq"} 0
tor_relay_dos_total{type="marked_address"} 0
tor_relay_dos_total{type="marked_address_maxq"} 0
tor_relay_dos_total{type="conn_rejected"} 0
tor_relay_dos_total{type="concurrent_conn_rejected"} 0
tor_relay_dos_total{type="single_hop_refused"} 0
tor_relay_dos_total{type="introduce2_rejected"} 0
# HELP tor_relay_load_onionskins_total Total number of onionskins handled
# TYPE tor_relay_load_onionskins_total counter
tor_relay_load_onionskins_total{type="tap",action="processed"} 0
tor_relay_load_onionskins_total{type="tap",action="dropped"} 0
tor_relay_load_onionskins_total{type="fast",action="processed"} 0
tor_relay_load_onionskins_total{type="fast",action="dropped"} 0
tor_relay_load_onionskins_total{type="ntor",action="processed"} 0
tor_relay_load_onionskins_total{type="ntor",action="dropped"} 0
tor_relay_load_onionskins_total{type="ntor_v3",action="processed"} 0
tor_relay_load_onionskins_total{type="ntor_v3",action="dropped"} 0
# HELP tor_relay_exit_dns_query_total Total number of DNS queries done by this relay
# TYPE tor_relay_exit_dns_query_total counter
tor_relay_exit_dns_query_total 0
# HELP tor_relay_exit_dns_error_total Total number of DNS errors encountered by this relay
# TYPE tor_relay_exit_dns_error_total counter
tor_relay_exit_dns_error_total{reason="success"} 0
tor_relay_exit_dns_error_total{reason="format"} 0
tor_relay_exit_dns_error_total{reason="serverfailed"} 0
tor_relay_exit_dns_error_total{reason="notexist"} 0
tor_relay_exit_dns_error_total{reason="notimpl"} 0
tor_relay_exit_dns_error_total{reason="refused"} 0
tor_relay_exit_dns_error_total{reason="truncated"} 0
tor_relay_exit_dns_error_total{reason="unknown"} 0
tor_relay_exit_dns_error_total{reason="tor_timeout"} 0
tor_relay_exit_dns_error_total{reason="shutdown"} 0
tor_relay_exit_dns_error_total{reason="cancel"} 0
tor_relay_exit_dns_error_total{reason="nodata"} 0
# HELP tor_relay_load_oom_bytes_total Total number of bytes the OOM has freed by subsystem
# TYPE tor_relay_load_oom_bytes_total counter
tor_relay_load_oom_bytes_total{subsys="cell"} 0
tor_relay_load_oom_bytes_total{subsys="dns"} 0
tor_relay_load_oom_bytes_total{subsys="geoip"} 0
tor_relay_load_oom_bytes_total{subsys="hsdir"} 0
# HELP tor_relay_load_socket_total Total number of sockets
# TYPE tor_relay_load_socket_total gauge
tor_relay_load_socket_total{state="opened"} 0
tor_relay_load_socket_total 0
# HELP tor_relay_load_tcp_exhaustion_total Total number of times we ran out of TCP ports
# TYPE tor_relay_load_tcp_exhaustion_total counter
tor_relay_load_tcp_exhaustion_total 0
# HELP tor_relay_load_global_rate_limit_reached_total Total number of global connection bucket limit reached
# TYPE tor_relay_load_global_rate_limit_reached_total counter
tor_relay_load_global_rate_limit_reached_total{side="read"} 0
tor_relay_load_global_rate_limit_reached_total{side="write"} 0
위 행이 실제로 뜻하는 바를 살펴보겠어요:
tor_relay_load_onionskins_total{type="ntor",action="dropped"} 0
중계기에 'dropped'(중단됨)이라는 글귀를 발견하셨다면, CPU나 RAM 쪽 문제인 경우가 일반적입빈다.
Tor is sadly single threaded except for when the "onion skins" are processed.
'onion skin'은 암호화기법이 적용된 작업으로, 잘 알려진 'onion 계층'을 대상으로 매 회로마다 적용돼야 작동해요.
Tor는 계층을 처리할 때 '스레드 풀'을 사용해요. 그리고 처리한 결과물을 스레드 풀에 위탁해요.
해당 스레드 풀에서 메모리나 CPU에서의 압박(pressure)으로 인해 작업이 중단될 수 있어요. 그럴 경우 과부하 상태가 작동(trigger)돼요.
서버의 모든 자원이 가동 중일 경우(at capacity), 과부하 상태가 작동될 가능성이 큽니다.
tor_relay_exit_dns_error_total{...}
Any counter in the "*_dns_error_total" realm (apart from the one for successful queries) indicates a potential DNS related problem.
However, we realized during the 0.4.7 release cycle that DNS errors are way too noisy and contain too many false positives to be useful for overload reporting purposes.
We therefore don't use them anymore for that purpose starting with 0.4.6.9 and 0.4.7.4-alpha. However, we still keep DNS metrics around to give the relay operator insight into what is going on with their relay.
DNS timeout issues and errors only apply to Exit nodes.
tor_relay_load_oom_bytes_total{...}
메모리 부족 발동(Out Of Memory invocation, OOM invocation)은 Ram 문제를 가리킵니다.
중계기의 Ram 용량 증설이 필요한 경우거나, 메모리 누수 문제일 수 있어요.
tor 프로세스에서 메모리 누수를 발견하셨다면, 해당 문제를 Tor gitLab로 제보해주시거나, Tor 중계기 메일링 리스트에 기재된 주소로 이메일을 보내주세요.
Tor엔 자체적인 OOM 핸들러가 있어요. Tor 추정 상의 가용 메모리 중 75%가 점유되는 순간 핸들러가 발동돼요.
Thus, let's say tor thinks it can use 2GB in total then at 1.5GB of memory usage, it will start freeing memory.
이러한 상태를 과부하 상태로 간주해요.
To estimate the amount of memory it has available, when tor starts, it will use MaxMemInQueues or, if not set, will look at the total RAM available on the system and apply this algorithm:
if RAM >= 8GB {
memory = RAM * 40%
} else {
memory = RAM * 75%
}
/* Capped. */
memory = min(memory, 8GB) -> [8GB on 64bit and 2GB on 32bit)
/* Minimum value. */
memory = max(250MB, memory)
과부하 상태를 방지하려면, 2GB 이상의 램이 장착된 64비트 시스템에서 중계기를 운영하는 걸 권장해요.
4GB의 램 용량이 바람직해요. 물론 더 증설한다고 시스템적으로 해가 되는 건 결코 아니에요만.
Note: If you are running a powerful server with lots of RAM then you might end up in an overloaded state due to the default queue size limit of 8GB even though you still have plenty of RAM unused. Add an appropriate MaxMemInQueues entry to your torrc configuration in that case.
Tor가 OS OOM 핸들러 자체에 의해 호출될 수도 있음을 신경쓰시는 분들도 계실 수 있어요.
이는 tor가 실행될 때 시스템의 총 메모리 용량을 가져오기 때문이에요. 시스템 전반에서 많은 애플리케이션이 램을 사용하면서 실행 중이라면, 종국에는 너무 많은 메모리가 점유되게 돼요.
이런 경우 OS는 OOM 트리거를 tor에 발동할 수 있어요. 이때 tor는 메모리 프레셔(memory pressure)를 인식하지 못한 상황이에요.
tor_relay_load_socket_total
If the number of opened sockets is close to or the same as total sockets available then this indicates the relay is running out of sockets.
tor 프로세스를 대상으로 ulimit -n 값을 늘리면 해결돼요.
tor_relay_load_tcp_exhaustion_total
다음 행은 중계기가 TCP 포트에서 실행 중임을 나타냅니다.
sysctl를 상기된 바와 같이 조정(tune)해 보세요.
tor_relay_load_global_rate_limit_reached_total
해당 카운터가 짧은 시간 동안 상당한 수치량으로 증가한다면, 중계기가 혼잡한 겁니다.
큰 어니언 서비스 또는 네트워크에서의 DDoS에 의해 지킴이로 사용되고 있는 것 같아요.
중계기가 여전히 과부하 상태이고 원인 규명이 안 됐다면 [email protected]에 메일을 보내주세요.
network-report OpenPGP key을 통해 메일을 암호화해 보낼 수 있어요.