Go to the SETUP > Netowrk page. Look toward the bottom and you will see the DHCP Addresses that have been handed out. There is a remove link you can use to get rid of them.
What do you have your address lease time out set to? Normally they will disappear on their own after the time out period has ended.
If you have reseved addresses, they will always appear even if the item hasn't connected.