Similar to what KAf says, the simplest setup you can get is one of the iOS devices. Ipad/iphone, all pretty much come with what you need to record. It really depends on what quality of recording you need to make.
If you just want to jot down song ideas, then you can get away with just an iphone or any other handheld recorder.