After a bit of tinkering with screen I found two major annoyances. First: X-forwarding does not work despite the -X flag being set when opening the ssh connection. Secondly: Shift-PgUp and Shift-PgDn do not work for scrolling (and scrolling is in general a pain).

The first problem seems to have a rather simple solution. Just set the DISPLAY environment variable in .bash_profile once a screen session has been detected.

if [ -n $WINDOW ] ; then 
  export DISPLAY=localhost:10
fi