What prevents me from setting up my own name server, and registering any domain I want, with whatever TLD I can come up with? For example
Its entirely possible to start up a local domain system - many companies do that internally, and there have been efforts to start an alternate root domain system with attempts at a parallel name registry system like Alternic and Opennic in the past. Neither worked out very well though.
Anyone who'd use it would need to have your own root name servers, or recursive name servers linked to that root name server as their domain name server - so your alternate name server system would be unreachable to the wider internet.
Many routers or networking setups do varients of this - zeroconf/bonjour use .local domains for systems on it, and windows uses hostnames without any domain on a workgroup.
Nothing prevents you from setting it up. Getting it used by others is tricky though. I suppose you COULD use 'regular' subdomains as an alternate to your own domain name systems (for example foo.bar on your DNS system would also point at foo.bar.shinydomain.com) or a proxy service of some sort - opennic does this.
If you were using both the regular and your own DNS, you'd also have to work out some way to handle conflicts, especially with the new TLDs coming up.
Nothing. But what would the point be? Anyone not using your nameservers wouldn't be able to resolve these names.