Please help, my relationship is sinking!?

This is a long story, so I will try & make it as short as possible. My partner & I have been together for 8yrs. We have 2 children together. We have had our ups & downs, but I think its coming to an end, again! 3 yrs ago we got engaged, and I destroyed the engagement by walking away. He works 60+ hrs per week, and was never home with me & our kids, I was pretty much raising our kids as a single parent. I turned to others on the internet for company ( which is no excuse) & it all fell apart. We got back together after loosing our house, and tried our relationship again. It was still the same. He was supposed to cut down on work, and be more family involved. He hasnt always been the family man type, but he does work hard to give our family nice things & provide for us. His mates also play a big part in his life, which I found hard & still do to handle. In November last year things still hadnt changed for the better, so I told him that I had enough, and he moved out. 2 weeks after he left I slept with a friend of mine, which is the biggest mistake I have ever made, its just not me at all. I hate myself for it now, but we are trying again, and have been since December to work it out. I have tried so hard in every way possible to show him & our kids how sorry I am for what I have done. It not only destroyed our family, but my friends aswell. I no longer have anything to do with this guy, and have no communication with him on any level. When my partner found out what I did, all hell broke loose, he was divistated, and all parties involved were ulted & charged. We have only just finished our court hearing because of this happening, which was very hard to go through, especially for my partner. He says he loves me & our kids very much, but he cant let the past go, and is finding it hard to forgive. I now live with my parents again since this happening, and he lives in shared accomodation with a friend of his. I want nothing more than him to come back home to us, I have been house hunting for the last 4 months trying to get our family back together again, but he says he not ready. we have been back together for 6-7 months now, does he not love me enough to know whether he wants our life back to normal, or am I expecting to much? I feel like my love and affection is not mutual, and I love & miss him dearly. Our kids miss him deeply also, & want him home. I am trying so hard in everyway, but I feel as im the only one trying. I see him nearly every night, but thats as far as it goes. Anytime I try talk about these concerns I have with him, he gets defensive. Alot of the time I will try & call or msg him, and he wont call back or reply, and its usually about our future. I fell pregnant with hin in late January, and have only just had the termination doen a few days ago. It was a 2 day proceedure, he came the 1st day, but I went alone the 2nd day, which broke my heart. I had surgery in March, and he wasnt there for me at all, and my Dad is dying from Cancer, and he's never ere for me, or here to support me. I just want my life back, I did the wrong thing, and im deeply sorry for what I have done, but I cant keep getting pushed away & punished all the time. I dont want to loose him again, but I cant keep going like this. Every time we argue this gets pushed in my face, I dont know what to do anymore. I have just told him I cant keep trying so hard anymore, im punishing myself, and I have had no reply. Which hurts so much. Do I leave him alone for a few days, or keep trying to tell him how I feel.? When we split in November he would call 20+ times a day saying how sorry he was for being a bad father & never here for us, and now we are back together he dosnt care anymore, he dosnt seem to miss us at all. I think he likes his lifestyle now, where has has no worry for our kids because they are with me full time, and he dosnt have to come home & answer to anyone or anyone being under his feet, he can come & go as he pleases. Please help me, what can I do? :((((