SSH works fine but rsync or scp hangs after a while which makes file transfers impossible. It turned out to be an MTU issue: setting it from the default 1500 to 1492 fixed the rsync/scp issue.
It happened to me again and having taken the first bullet now I decided to figure out what's happening behind the scenes.
Every network link has a maximum packet size called the link's MTU (Maximum Transmission Unit). The full path from one computer to another may travel across many links with different MTUs. The smallest MTU for all the links in a path is the path MTU.
The MTU and the packet should be as large as possible, in other words, the least fragmented. I found an RFC from 1990 about Path MTU discovery which describes how we can make the packet as large as possible.
We have to find out the smallest MTU along the links. The basic procedure is simple: send the largest packet you can, and if it won't fit through some link will get back with a notification saying what size will fit.
Here comes the catch: the notifications arrive as ICMP (Internet Control Message Protocol) packets known as "fragmentation needed" ICMPs.
Some network and system administrators view all ICMPs as risky and block them all, disabling path MTU discovery, usually without even realizing it. Then if the default MTU happens to be larger than the path MTU we are out of luck unless we manually change it.