Slow performace & memory occupation when using SQL 2008 64 bit on HP DL120G7 with 16GB Memory

I am using SQL Server 2008 64bit on a HP DL120G7 with 16GB memory.

when the server starts up the occupied memory is about 6GB & it gradually goes up to 16GB in 3~4 hours.

So the performace falls down dramatically.