Well, did you happen to notice MessageTrace>>#deselectAll? It works but not hooked up to a menu option so you'd have to inspect the MessageTrace model and send it from there.
Also, have a look at MessageTrace>>#toggleSelectionAt:shifted:controlled:. Since 2002 (that's 14 YEARS!), it has been designed to operate like windows selection: Shift to extend or reduce the selection range -- that actually still works -- and Control to toggle individual selections, which no longer works because Control forces the "Control-Menu" because, I assume, Apple's only have one mouse button..? I'm not sure maybe Bert knows..?
One thing I'm curious though -- you want to file out methods you're selecting via script, so why not just file them out with your script rather than invole UI machinery in that? The UI is optimized for *user interaction*, not one-off batch operations..