Thanks, it now works. Strangely enough when I click up in the keyboard when the Vertical scrollbar is selected it actually moves down and the other way around (hitting down goes up). Any ideas of what could this be?
EDIT: I just solved it, silly me. Changing direction to bottomToTop did it.
EDIT2: Turns out it is NOT SOLVED, changing the direction to bottomToTop allows me to control it with keyboard as I want but I would like the scrollbar to be topToBottom.