Wed, Jun 03, 2015 at 10:13:16PM CEST, gslavic@sox.rs wrote:

Dear Mr Jiri Pirko,

           My name is Goran Slavić, and I work for Serbian Open

eXchange. We have setup the teaming on 4x10Gb/sec ports of our routing server. Unfortunetly load balancing between teamed ports doesn't work very well (we have extremely UN-balanced traffic on teamed ports). Traffic analysis suggests that the balancing is only done with MAC addresses and the rest of configuration directives ("ipv4, ipv6, l4" ...) is ignored/not taken in consideration when load balancing traffic between ports !

           You can find a detailed explanation (what we configured on

ports/teaming software), our configuration details and graph that are the result of our testing on this link (forum where we first shown the problem and asked people to help us):

           https://www.centos.org/forums/viewtopic.php?f=50&t=52121

           We would highly appreciate any help, advice or suggestion

you can give us regarding this problem.

Hello Goran.

It's always good idea to cc libteam mailing list for reports such as yours.

TX port selection is done based on hash (0-255) computed by BPF function. Then, the hash is either used directly to determine tx port (hash%number_of_ports), or there is a hash-port mapping where every hash is mapped to a tx port. The second case is yours.

You can see the mapping either in teamd debug output, or if you list team options directly by "teamnl team0 options". With mapping there are also statistics for each hash there.

So in your case, there might be 3 issues: 1) your traffic is not differenciated enough, so most of the packets falls into sigle hash. That would happen for example for one tcp stream between 2 ip addresses. 2) hash function fails to produce good hash according to the packets coming through. You could try to dump the BPF function and look at the inctructions by hand. Or another thing would be to assebmle your own hash function and push it into kernel manually (using teamnl team0 setoption bpf_hash_func BFPCODE) 3) mapping is wrong. Balancer might do the mapping wrongly. To figure this out, the debug messages or options dump will help you to find out if this is the case.

Please gather more info so we can proceed to debug this.

Thanks.

Jiri