Metals are the elements that occur naturally to environment in form of rock, soil or water. If high concentration of metals was detected in the environment it will cause bad effects to the surrounding. In this study, metals concentration in sediment from Besut and Dungun River were investigated. The objectives of this study are to determine the concentration of metals in sediment and to evaluate sources and level of metal pollution using pollution assessment such as Enrichment Factor (EF) and Geoaccumulation Index (Igeo) in Besut and Dungun River. Seven sediment samples from Besut River and three from Dungun River were dried and sieved by using 75µm sieves. Sediment samples were analysed by X-ray fluorescence. Based on the mean values of the metal concentrations, the metals follow in the following order, from the highest to the lowest; Si > Fe > Al > K > Ti > Zr >Ca> Mn. When the EF value is calculated, the results showed that Fe, Mn, K, Ti, and Zr were come from anthropogenic sources and Al, Ca, and Si were come from natural sources. The results was supported by Igeo when it was found that both rivers were moderately polluted to extremely polluted by Fe, Mn, K, Ti, and Zr. Most of the pollution shown that the sources were come from point sources such as activities nearby to a river bank, water gate, road bridge and aquaculture at the T junction of a river.