I have a welsh section A pony and he is nervous any suggestions?

I bought a welsh pony last fall and he was halter broke and that is it. He was a stallion but I just had him castorated a couple of days ago. I have done lots of ground work with him but he is still nervous. I keep him locked up so I have to catch and touch him every day. He will be good for a few days then he acts like I haven't touched him for days. He is not mean at all and he knows he is to be by me and not run away but you can tell sometimes it takes all he has to stay there and not flee. Some friends have told me to trip him and that might give him a wake up call but I am not sure. I have worked with horses who have never been touched and they were alot better then he is. I like him and he is kind of flashy but if anyone has advice on whether I am waisting my time or if he might come around. He is supposed to be a kids horse, well there is no way I would put my kids on him right now.Oh! Some things like being covered with a tarp he was better than another but he has some issues.