I've tried it on two different machines, in two different ways. All of the software I mention below is free.
On the first, I installed the Haali Media Splitter to recognize the .mkv extension, and the "ffdshow-tryouts" codec to decode the H.264 video stream inside. It plays fine in Windows Media Player and in Media Center, but on the DSM-750 I jusst get a black screen with sound.
On the second, I installed the new DivX 7 codec, which comes with both the .mkv splitter and the H.264 codec. It played in WMP at that point but Media Center still couldn't see the file, so I had to manually add the .mkv extension to the registry (a google search will show you how to do this). After that it showed up and played in Media Center just fine, but still just a black screen with audio on the Extender.
So, uh, Dlink? Any help here? Has anyone got it working?