Robert Sedgewick of Boston had been ordered by Robert Cromwell to attack New Holland (New York).  But after he got everything ready, a peace treaty was signed between the English and the Dutch.  Since he was "all dressed up with nowhere to go," he attacked Acadia in August 1654 and destroyed most of the settlements (even though it was peacetime) ... including Port Royal, La Have, and the Saint John River.   He left the area, but had appointed an Acadian council with Guillaume Trahan in charge. [Clark, p. 113]

     The seigniorial system was still going strong.  LeBorgne was still making his claim against Denys and LaTour.  When Denys got his royal grant to the St. Lawrence area in 1654, LeBorgne had to give St. Peters and Nipisiguit back to him. A letter by Meneval on Nov. 7, 1689 shows that Belle Isle was the seigneur of most of the Port Royal area. 
     Over the years, the Acadian seigneurs like LeBorgne really didn't live up to their part of the deal.  They were supposed to bring settlers to the New World and to take care of them. There's no record of them building mills or bake-ovens.  They basically just tried to collect rent (cens et rentes).  Corvees or charging for fishing or cutting lumber are not mentioned.  [Clark, p. 120]  The system ended (as far as the British were concerned) when Agathe de LaTour married a British soldier named Campbell in 1733 and received 2000 pounds for her "rights." [Clark, p. 119]
     The Acadians weren't exactly the best tenants.  They'd pay token amounts unless heavily pressured.   The seigneurs usually got little for the land.  The Acadians were almost like freeholders, and paid little taxes.  A map of the seigneuries was done in 1700 and sent to France, but no one can find it. Since their days in France, the Acadians  thought that the only way to have land was get it from a seigneur, who got it from the king.  Acadians also probably thought the land they farmed was the seigneur's, and therefore the king's.  In Acadia, grants were poorly described and often overlapped, leading to conflicts. [Clark, p. 114]    We find numerous records of legal cases brought by the Acadians;  they, like the Canadians, were very litigious. 

1657
    In 1657 or 1658, Sir Thomas Temple came to Acadia aboard the Satisfaction.  He was given the governorship by Commonwealth authority in 1657 (which was confirmed after the Restoration in 1662).  His grant of Acadia came from Cromwell, and was shared by LaTour and William Crowne.  He kicked out some of LeBorgne’s men at LaHave in 1658 and some French fishermen at Port Rossignol in 1664.  Temple established his presence at Rossignol and at Mirligueche (Lunenburg), but Temple's control of the colony probably only extended to the immediate neighborhood of Port-Royal and a few other sites. Temple himself lived in Boston and seldom visited Acadia. [Clark, p. 107]
     We find an Order of the Council of State (Apr. 14, 1657) that states: 
            A convoy to be provided for several ships bound to Newfoundland, and instructions given to the 
                commander to make one of them ready with all speed to carry Col. Thomas Temple and his company 
                to his plantation in Nova Scotia or Acadia, in order to his settling in the forts and government there, 
                according to his patent and commission from his Highness.
                                                                                                          [Interregnum, Entry Bk., Vol. CV., p. 790]
     Also, there is a message (Nov. 12, 1657) from Capt. Peter Butler of the Satisfaction to the Navy commanders. 
              After receiving Col. [Thos.] Temple and his company on board, sailed for Boston, New England, and
              then made for St. John's Fort and Port Royal, intending for Newfoundland; but meeting with violent
              storms, and getting short of provisions, returned for England; neither he nor his company are ashamed
              to speak of the goodness of the Lord in preserving them from such great dangers.
                                                                                                          [References. Vo.  174. 69]
     Supposedly, there was a list of colonists of 1658 that was around in the 1950s, but it cannot be found.   Dulong discusses the missing list at his Michel Forest webpage [Dulong, <http://fp-www.wwnet.net/~dulongj/Forest.htm>] 
     According to a document in the Boston archives, Pierre LaVerdure, his wife Priscilla Mellanson, and their sons may have been aboard that ship.  Though the parents and possibly one son were thought to have moved on to New England, two of the sons ... Charles and Pierre ... stayed in Acadia.  They adopted their mother's name (Melancon/Melanson).  When the Grand Pre area was settled, Pierre was a captain and a leader in that community.  In fact, government orders were sent through him. 
     Other settlers that may have settled in this period were Laurent Granger and Roger Caissy. 

     Though the English didn’t make a real impact on Acadia, it did produce a time where the settlers had little contact with the French, but more contact with the New Englanders.  Some learned a bit of English to communicate.  They learned to get along with the English.  And those born in this period were those in charge once England took over for good in 1710.  Many of the first generation of Acadians were born at this time. [Daigle, p. 23]

1666
     In 1666, France made the decision not to send more colonists to the New World.  They thought it “would not be prudent to depopulate its kingdom to populate Canada.” [PAC, C11A, 2:199, “Colbert a Talon”, Jan. 5, 1666]  Meanwhile, many English were still making the trip to avoid religious persecution and for economic success.  In 1670, Acadia had about 400 people, while Massachusetts had 40,000. 

1667
     France regained Acadia after the Treaty of Breda in 1667.  LeBorgne’s son Alexandre 
LeBorgne de Bellisle became governor from Canso/Baie Verte to New England.  Temple caused enough problems and delays that the official change of hands didn't come till 1670. 

1670
      Hector d’Andigne de Grandfontaine received the surrender of the English forts at Port Royal and Jemseg in 1670, and of the Cape Sable area a bit later.  Some of the French may have headed back to France back in 1654. [Clark, p. 108]
     Most of the 1670 population had been born in Acadia.  The Minas settlement started between the Riviere St. Antoine (Riviere des Habitants/Cornwallis River) and the Gaspereau River.  It spread to beyond the Canard, Petit Habitant, and Pereau to the foot of the North Mountain.  It also extended to Pisiguid and Cobequid.  At Chignecto Bay, settlement began at Beaubassin and spread to the Missaguash, La Planche, Hebert, and Maccan, Aulac, Tantramar, Shepody, Petitcodiac, and Memramcook rivers, the marsh of the Minudie peninsula, etc. [Clark, 109]