The agency said the boy was initially taken to at a New Mexico hospital where he was diagnosed with a common cold, with a fever reaching 103 degrees Fahrenheit, and was prescribed medication.
The boy was returned to the same hospital after starting to vomit.
CBP says it does not know the cause of the child's death. Democrats in Congress and immigrant-rights activists across the country have said the blood of children is on the government's hands, insisting bad policies and poor attitudes contributed to the deaths.
CBP Commissioner Kevin McAleenan said Wednesday the agency has more than 1,500 emergency medical technicians on staff and that officers are taking dozens of sick children to hospitals every day.
On Tuesday, Rep. Castro offered his condolences to the boy's family and called for a congressional investigation of the death.
It was the second death of a child in the agency's care this month.
The boy, who was detained with his father, died shortly before midnight at Gerald Champion Regional Medical Center in Alamogordo, New Mexico, about 90 miles north of the border crossing in El Paso, Texas.
Meghan Markle's Christmas Day look was all from Victoria Beckham's line
The Duchesses of Cambridge and Sussex have been dogged by rumours they aren't bosom buddies for months. Many had brought bouquets of flowers or had got in the Christmas spirit by wearing Santa hats.
China says detention of Canadian involves employment issue
A Canadian lawmaker said Wednesday that the case involved visa irregularities for an Alberta woman who had been teaching in China. Canadians who study or do business in China may want to lie low and keep a low profile, she said. "One detention is bad enough".
Washington safety Montae Nicholson done for season, after video of fight surfaces
The Loudoun County Sheriff's Department arrived and arrested Nicholson after the attack on Tuesday, December 18. He has not played in the last four games and has all but played his last game in a Redskins uniform.
In the wake of the boy's death, McAleenan announced a series of moves on Tuesday night.
The agency said in a statement that one of their agents saw the boy had a cough and "glossy eyes" on Monday evening, prompting him to be hospitalized on Christmas Eve.
The Guatemalan foreign ministry called for an investigation "in accordance with due process". They were taken to the El Paso Border Patrol station Thursday and moved to Alamogordo, N.M., at 1 a.m. Sunday "because of capacity levels at the El Paso station.
This is why I asked Congress on Tuesday to change our laws so that the United States is not incentivizing families to take this risky path", McAleenan said.
"You hear that in the United States they pay well, not like the companies here in Guatemala - that's why people leave", said Pop, whose 22-year-old son Joaquin left in November in search of the American dream. The border agents later had to clean the boy's vomit, but the father "declined further medical assistance" for the boy.
The news triggered outrage on social media, days after the body of a Guatemalan migrant girl who died in similar circumstances was returned home.
Agency guidelines say immigrants generally shouldn't be detained for more than 72 hours in CBP holding facilities, which are usually smaller and have fewer services than ICE detention centres.
Her death on December 8 prompted calls for investigation from the Congressional Hispanic Caucus, which says "disturbing systematic failures" and a lack of medically trained agents prevented the government from adequately caring for its child. She was flown by helicopter to a hospital in El Paso, Texas, where she died. The Guatemalan government did not immediately respond to a request for comment.