Having done some digging I found this on the TfL website even though TfL
have done some "tidying up" since the new Overground link opened.

http://www.tfl.gov.uk/assets/downloa.../south-london-
line-study-findings.pdf

A couple of the options show LOROL running into Victoria. It has not
proved possible to make any of these options happen due to political
disagreement on a monumental scale. The DfT even refused to allow a few
long distance trains to stop at Peckham and Denmark Hill. The whole
thing has been an unedifying mess which does passengers very few
favours. I will be astonished if TfL are able to get their most recent
proposal of a Bromley South - Victoria off peak service in for the 2014
timetable change.

I am sure I have read of a proposal for LOROL to have "emergency"
running rights into Victoria in the event of engineering works.
However I think even this was turned down which is why LOROL would
divert to Battersea Park and turn there.

Ta - that was indeed an interesting read - it seems the big quibble with
using Victoria for LOROL is the fact it's only 4 carriage - which I
suppose can be used better in other ways....