The Ten Commandments, which Sabbath is part of, were not given until Exodus 16.
Joseph knew that it was a sin to commit adultery in Genesis 39:9, which is one of the Ten Commandments, so your claim is demonstrably false.
Not Abraham, not Isaac, not Jacob, none of the patriarchs kept Sabbath (Fathers prior to Egypt did not keep the Sabbath:
Deut 5:2-3 "The Lord our God made a covenant with us at Horeb. The Lord did not make this covenant with our fathers, but with us, with all those of us alive here today."). The first time the Sabbath is mentioned in some significant way is in the 16th chapter of Exodus, when God feeds the people manna from heaven as they wander in the wilderness.
God made the New Covenant with us that he did not make with the ancient Israelites, but that does not mean that any of the commands in the New Convent were not previously given, so the fact that God made a covenant with the ancient Israelites that He did not make with their fathers does not mean that any of its commands were not previously given.
And the manna comes every day except the Sabbath day, and the day before they get enough for that day, so that they don’t have to work on that day. And that gives them a little preview of what’s coming, because in the 20th chapter you have the Ten Commandments, and in the Ten Commandments, prescriptions are given that do set down laws for the Sabbath day. This is the first time any such laws have been given by God.
The fact that Exodus 20 is the first recorded instance of God commanding against adultery does not mean that it is the first time that God commanded against it.
The Sabbath was not instituted for man in Genesis. It was instituted officially in Exodus, in the law of Moses. A further understanding of that comes from Exodus chapter 31. The Lord speaks to Moses in verse 12, and He says to him, “As for you, speak to the sons of Israel saying, ‘You shall surely observe My Sabbaths; for this is a sign between Me and you throughout your generations, that you may know that I am the Lord who sanctifies you.
“‘Therefore you are to observe the Sabbath, for it is holy to you. Everyone who profanes it shall surely be put to death; for whoever does any work on it, that person shall be cut off from among his people. For six days work may be done but on the seventh day there is a Sabbath of complete rest, holy to the Lord; whoever does any work on the Sabbath day shall surely be put to death. So the sons of Israel shall observe the Sabbath, to celebrate the Sabbath throughout their generations as a perpetual covenant.
“‘It is a sign between Me and the sons of Israel forever;’” - why? - “‘for in six days the Lord made heaven and earth, but on the seventh day He ceased, and was refreshed,’” - or rested. Here we find that Sabbath is a sign; it is a sign. That is to say, it points to something else. It is a symbol, connected to the Mosaic covenant.
When God made a covenant with Noah, He promised Noah that He would never destroy the world again, and God identified a sign. What was the sign of the Noahic covenant? Rainbow. When God made a covenant with Abraham, He made that covenant with Abraham and He designated a sign, and the sign of the Abrahamic covenant, participation among the covenant people Israel, was the sign of circumcision. And here you have in the Mosaic covenant another sign, and the sign this time is the Sabbath.
The fact that God gave the Sabbath as a sign does not mean that it was not previously a sign. If you think that the things that the Sabbath is a sign for are true, then you should live in a way that testifies about their truth by following Christ's example of keeping the Sabbath holy rather than a way that bears false witness against those things.
It was only a sign. Observing it with a duplicitous heart gained nothing. In fact,
Isaiah 1:13 says, “Bring your worthless offerings no longer, Incense is an abomination to Me. New moon and Sabbath.” The prophet Hosea pronounces a similar judgment on their hypocritical Sabbaths: “I will put an end to all her gaiety, her feasts, her new moons, her Sabbaths.” It didn’t mean anything to observe it outwardly without a heart of love and devotion to God.
In Exodus 20:6, God wanted His children to love Him and obey His commandments, so obedience to God has always been a matter of the heart.
The Sabbath was a reminder of creation. The Sabbath was to remind the people of Israel that they had forfeited paradise; that man had forfeited paradise. The law said to them, “Obey this law and you will be blessed.” God said that repeatedly: “Obey this law and you will be blessed,” to show them that righteous behaviour would restore a taste of Eden’s paradise. Righteous behaviour would also point to a future, a future kingdom when paradise would be regained.
The Bible states that the Sabbath is a memorial of when God rested after Creation, but it does not say that it is a memorial that Israel had forfeited paradise.
So, the Sabbath, every Sabbath that went by, when they rested, they were reminded of a perfect creation, a paradise of God dominated by righteousness, which had been forfeited by sin and could only be regained again by righteousness. God then institutes the seventh-day system - not for everybody in the world; in fact, specifically, it says, for Israel. Verse 17: “A sign between Me and the sons of Israel forever.” Every seventh day was a reminder that they were living in a fallen world. Every seventh day was a reminder that they had lost paradise.
In Ephesians 2:12-19, Gentiles were at one time separated from Christ, alienated from Israel and strangers to the covenants of promise, and without hope and God in this world, all of which is in accordance with Gentiles at one time not being doers of God's law, but through faith in Christ all of that is no longer true in that Gentiles are no longer strangers or aliens but are fellow citizens of Israel along with the saints in the household of God, all of which in accordance with Gentiles becoming does of God's law, so Gentiles become joined to Israel through faith in Christ.
The Sabbath was the sign to Israel of the Old Covenant. Because we are now under the New Covenant, we are no longer required to keep the sign of the Old Covenant. The New Testament nowhere commands Christians to observe the Sabbath.
In Matthew 4:!5-23, Jesus began his ministry with the Gospel message to repent for the Kingdom of God is at hand, which was a light to the Gentiles, and God's law was how his audience knew what sin is (Romans 3:20), so repenting from our disobedience to it is a central part of the Gospel of the Kingdom, which includes repenting from breaking the Sabbath. Jesus also set a sinless example for us to follow of how to walk in obedience to God's law and we are told to follow his example (1 Peter 2:21-22) and that those who are in Christ are obligated to walk in the same way that he walked (1 John 2:6), which includes keeping the Sabbath holy. So Christ spent his ministry teaching his followers to obey God's law by word and by example and the reason why he established the New Covenant was not in order to nullify anything that he spent his ministry teaching or so that we could continue to have the same lawlessness that caused the New Covenant to be needed in the first place, but rather the New Covenant still involves following God's law (Jeremiah 31:33, Ezekiel 36:26-27).
Paul warned the Gentiles about many different sins in his epistles, but never about breaking the Sabbath.
Jesus quoted three times from Deuteronomy in order to defeat the temptations of Satan, which included saying that man shall not live by bread alone but by every word that comes from the mouth of God (Deuteronomy 8:3), so he affirmed God as being an authoritative source, which includes affirming what God spoke in Deuteronomy 5:12-15 in regard to keeping the Sabbath holy, and we have no need for Paul to repeat everything that God has spoken in order to know that we should still obey God.
The early church Fathers, from Ignitions to Augustine, taught that the Old Testament Sabbath had been abolished and that the first day of the week (Sunday) was the day when Christians should meet for worship.
So when God has commanded something and the early church commentators taught to rebel against what God commanded, then who has the highest authority and which one should you follow?