There isn't a really good way to do this, because in order to get to that directory, other users will need to be able to get into MyDir, MyDir/Library, etc. That said, there are a couple of ways to sort-of accomplish it:
You could set the permissions on MyDir, MyDir/Library, etc to allow traversal, but not reading. Problems with this approach: other users won't be able to navigate to the folder normally, they'd have to use the Finder's Go to Folder option (or Cmd+Shift+G) and type in the path to it. Also, although they wouldn't be able to see the contents of MyDir, MyDir/Library, etc, they would be able to access the files there if they new (/could guess) the names. Since the contents of Libraries tend to be pretty predictable, this means they'd be able to get at MyDir/Library/Preferences, MyDir/Library/Keychains, etc unless you properly protected those as well. Assuming you wanted to do this, here are the relevant setup commands (warning: these are not easily reversible, please think carefully before executing):
chmod og= MyDir/* MyDir/Library/* MyDir/Library/Application\ Support/*
chmod og=x MyDir MyDir/Library MyDir/Library/Application\ Support
chmod og=rx MyDir/Library/Application\ Support/Games
You could actually put the contents somewhere more accessible, and make MyDir/Library/Application Support/Games a symlink to it. Potential problems with this come from programs that don't follow symlinks (especially, installers that might replace it with an actual directory -- you'd have to clean up after that one by hand). Here's an example, using /Users/Shared/Games as the public location:
mkdir /Users/Shared/Games
ln -s /Users/Shared/Games MyDir/Library/Application\ Support/Games
