How Can I Help This To Work?
First I want to say I love him so very much. I love his daughter, my SD so very much. But I'm coming to a crossroads and I don't want to be there.
My SD alternates weeks between our home and her BM. My BF has panic attacks 4 days prior to her return. I'm guessing in anticipation of us fighting and we might end up fighting. He denies fault for picking fights and he has to be right no matter what. This wasn't always the case. My SD has lived with us since we've been together and since her birth been with her father, my BF. I've asked him to just bring her home. He's said no. I think it was a little better when she was home. Now he keeps picking and it takes a day or two for him to calm down enough to listen to reason. Until then I am silent and I don't interact with him a lot and just keep to myself. That's hard in itself since I'm a physical person and want to be around him and touch him. During these times I can't sleep at night and I'm just unhappy all the way around. I'm getting to the end of my rope but I don't want to let go because I do love my BF and my SD. 
How old is your SD and is she
How old is your SD and is she still coming over every other week? When you say your BF is afraid you will fight when he brings her home is that the way things have gone? Are the fights about your SD or other things? Are you fighting with your BF and SD?
My SD is 9 years old. In the
My SD is 9 years old. In the past she's admitted to behaving badly and being mean to break us up. Because we typically only fight about the kids. And that's all of the kids not just one or the other. Mostly discipline and treatment of the children. I'm fighting with my BF. Yes we have fought a lot about this in the past. But he sent her over there every other week to see if we would fight less. And we do fight less, but he still has the panic and I can't help him. He won't tell me and he doesn't know what he's doing until it's all the way into a full blown fight. I don't fight with the children. Well, maybe a teenager every once in a while.