there is a lot of necessary information stored in the TMD. messing up any number of things will cause a title not to be able to be booted. and depending on the error handling in program which is trying to do the booting, you will get different results.
scenario 1 - you messed with the TMD before trying to install it. most likely you broke the hash/signature and it will fail to install ( the ES module in IOS says no way dude )
scenario 2 - the title is already installed and you read the TMD from the nand, changed the IOS to an invalid one, and wrote the file back to NAND. when anything tries to boot this title, it will try to boot a non-existant IOS. 99-100% of the time that results in a blackscreen. if this was your system menu, it would be a brick.
scenario 3 - same as above, but you edited one of the TMD content cids instead of the IOS. if this is cid # 0 of a channel, the system menu will not be able to display the banner, and either will not display this channel or will banner brick. if this is not a channel, but some other title, that title will fail to boot and result in a blackscreen. if this is something other than cid # 0, the title will start to boot correctly, but will fail when it tries to load the invalid .app file. this would either blackscreen or give a partially working title
scenario 4 - you edit the boot index field. same as scenario 3.
scenario 5 - you edit the size of a TMD content. this one will ok in some situations. the filesize is stored in the metablocks of the nand as well as the TMD. programs which read the size from nand will get the correct size. program that read the size from TMD will get the incorrect size. if the size is too small, they will think that .app file is smaller than it really is, which may result in a banner brick, blackscreen, ...
as of right now, i think everything else in the TMD should be ok to edit even after the title is installed on the nand. there is not any checking of sizes/hashes/signatures or anything after install time. however, there is always a chance nintendo will change this in the future, and if they do that, messing with the TMD will likely result in a blackscreen, or looking at their method, a "error #005" message.