Based on that scripture alone, everyone was different shade of brown before the so-called White men ever came to existence. And the so-called White men science backed it up also. 

Genesis 2:7

“And the LORD God formed man of the dust of the ground and breathed into his nostrils the breath of life; and man became a living soul.”

Science

"First modern Britons had 'dark to black' skin, Cheddar Man DNA analysis reveals" Source: First modern Britons had 'dark to black' skin, Cheddar Man DNA analysis reveals | Genetics | The Guardian

The so-called White men first appearance in the Earth.

Genesis 25:21-34

21 And Isaac intreated the LORD for his wife, because she was barren: and the LORD was intreated of him, and Rebekah his wife conceived.

22 And the children struggled together within her; and she said, If it be so, why am I thus? And she went to enquire of the LORD.

23 And the LORD said unto her, Two nations are in thy womb, and two manner of people shall be separated from thy bowels; and the one people shall be stronger than the other people; and the elder shall serve the younger.

24 And when her days to be delivered were fulfilled, behold, there were twins in her womb.

25 And the first came out red, all over like an hairy garment; and they called his name Esau.

26 And after that came his brother out, and his hand took hold on Esau's heel; and his name was called Jacob: and Isaac was threescore years old when she bare them.

27 And the boys grew: and Esau was a cunning hunter, a man of the field; and Jacob was a plain man, dwelling in tents.

28 And Isaac loved Esau, because he did eat of his venison: but Rebekah loved Jacob.

29 And Jacob sod pottage: and Esau came from the field, and he was faint:

30 And Esau said to Jacob, Feed me, I pray thee, with that same red pottage; for I am faint: therefore was his name called Edom.

31 And Jacob said, Sell me this day thy birthright.

32 And Esau said, Behold, I am at the point to die: and what profit shall this birthright do to me?

33 And Jacob said, Swear to me this day; and he sware unto him: and he sold his birthright unto Jacob.

34 Then Jacob gave Esau bread and pottage of lentiles; and he did eat and drink, and rose up, and went his way: thus Esau despised his birthright.

Esau became the nation of Edom

Genesis 36:8

Thus dwelt Esau in mount Seir: Esau is Edom.

Esau was a cave dweller.

Joshua 24:4 

And I gave unto Isaac Jacob and Esau: and I gave unto Esau mount Seir, to possess it; but Jacob and his children went down into Egypt.

Genesis 36:9

9 And these are the generations of Esau the father of the Edomites in mount Seir:

10 These are the names of Esau's sons; Eliphaz the son of Adah the wife of Esau, Reuel the son of Bashemath the wife of Esau.

11 And the sons of Eliphaz were Teman, Omar, Zepho, and Gatam, and Kenaz.

12 And Timna was concubine to Eliphaz Esau's son; and she bare to Eliphaz Amalek: these were the sons of Adah Esau's wife.

13 And these are the sons of Reuel; Nahath, and Zerah, Shammah, and Mizzah: these were the sons of Bashemath Esau's wife.

14 And these were the sons of Aholibamah, the daughter of Anah the daughter of Zibeon, Esau's wife: and she bare to Esau Jeush, and Jaalam, and Korah.

15 These were dukes of the sons of Esau: the sons of Eliphaz the firstborn son of Esau; duke Teman, duke Omar, duke Zepho, duke Kenaz,

16 Duke Korah, duke Gatam, and duke Amalek: these are the dukes that came of Eliphaz in the land of Edom; these were the sons of Adah.

17 And these are the sons of Reuel Esau's son; duke Nahath, duke Zerah, duke Shammah, duke Mizzah: these are the dukes that came of Reuel in the land of Edom; these are the sons of Bashemath Esau's wife.

18 And these are the sons of Aholibamah Esau's wife; duke Jeush, duke Jaalam, duke Korah: these were the dukes that came of Aholibamah the daughter of Anah, Esau's wife.

19 These are the sons of Esau, who is Edom, and these are their dukes.

20 These are the sons of Seir the Horite, who inhabited the land; Lotan, and Shobal, and Zibeon, and Anah,

21 And Dishon, and Ezer, and Dishan: these are the dukes of the Horites, the children of Seir in the land of Edom.

22 And the children of Lotan were Hori and Hemam; and Lotan's sister was Timna.

23 And the children of Shobal were these; Alvan, and Manahath, and Ebal, Shepho, and Onam.

24 And these are the children of Zibeon; both Ajah, and Anah: this was that Anah that found the mules in the wilderness, as he fed the asses of Zibeon his father.

25 And the children of Anah were these; Dishon, and Aholibamah the daughter of Anah.

26 And these are the children of Dishon; Hemdan, and Eshban, and Ithran, and Cheran.

27 The children of Ezer are these; Bilhan, and Zaavan, and Akan.

28 The children of Dishan are these; Uz, and Aran.

29 These are the dukes that came of the Horites; duke Lotan, duke Shobal, duke Zibeon, duke Anah,

30 Duke Dishon, duke Ezer, duke Dishan: these are the dukes that came of Hori, among their dukes in the land of Seir.

31 And these are the kings that reigned in the land of Edom, before there reigned any king over the children of Israel.

32 And Bela the son of Beor reigned in Edom: and the name of his city was Dinhabah.

33 And Bela died, and Jobab the son of Zerah of Bozrah reigned in his stead.

34 And Jobab died, and Husham of the land of Temani reigned in his stead.

35And Husham died, and Hadad the son of Bedad, who smote Midian in the field of Moab, reigned in his stead: and the name of his city was Avith.

36And Hadad died, and Samlah of Masrekah reigned in his stead.

37And Samlah died, and Saul of Rehoboth by the river reigned in his stead.

38And Saul died, and Baalhanan the son of Achbor reigned in his stead.

39And Baalhanan the son of Achbor died, and Hadar reigned in his stead: and the name of his city was Pau; and his wife's name was Mehetabel, the daughter of Matred, the daughter of Mezahab.

40And these are the names of the dukes that came of Esau, according to their families, after their places, by their names; duke Timnah, duke Alvah, duke Jetheth,

41Duke Aholibamah, duke Elah, duke Pinon,

42Duke Kenaz, duke Teman, duke Mibzar,

43Duke Magdiel, duke Iram: these be the dukes of Edom, according to their habitations in the land of their possession: he is Esau the father of the Edomites.

Jeremiah 49:22

Behold, he shall come up and fly as the eagle, and spread his wings over Bozrah: and at that day shall the heart of the mighty men of Edom be as the heart of a woman in her pangs.

Obadiah 1:4

“Though thou exalt thyself as the eagle, and though thou set thy nest among the stars, thence will I bring thee down, saith the LORD.”

Malachi 1:4

Whereas Edom saith, We are impoverished, but we will return and build the desolate places; thus saith the LORD of hosts, They shall build, but I will throw down; and they shall call them, The border of wickedness, and, The people against whom the LORD hath indignation for ever.