Field recording has always been a reflex for me, since I was about 15 years old. I began my field recording journey with cassette tape, when I bought an EIKI cassette recorder and player (highly recommend, these things are built like bricks) from eBay. They run off a whopping six C batteries, or can be plugged into a wall outlet. I ran mine off batteries, and ventured into the nearby San Antonio, TX wilderness (a nearby park). I recorded streams, birds, leaves, and shouting of soccer players, all with the on board microphone. Not knowing much about recording at the time, I purchased a very silly 1/8" microphone for the recorder, which allowed some directionality to my recording. I was able to get smaller sounds, along with some interviews. These sound sources were used for some very early works of mine, which solely relied on making cassette tape loops, a very grueling process to those familiar. I made "studio" recordings with these loops, as well as accompaniment by small synthesizers and Hewlett Packard oscillators.
I now have a much more modern field recording setup although I do occasionally pull out my one of two EIKI recorders. I currently operate with a Zoom H4n, which can record in up to 192kHz sample rate, as well as 24-bit. The H4n has a stereo pair up top and two XLR inputs, which I typically have a ribbon mic plugged into. This setup allows clear, high quality recordings which I typically use for sound pieces in ChucK or Max MSP/Pd. This method is not as clunky as the EIKI, and doesn't have the sonic touches of baked cassettes, but having the option is nice.
Field recording at a young age in a state that typically concerns itself with privacy and the right to it, ethics came into question quite early. I didn't want to provoke anyone, so I typically stuck to spaces that weren't occupied by humans (empty parks, the sides of highways, etc.). When field recording takes the shape of an artistic practice, I think the recorder must take into account the sounds their recording. This was a topic that arose in my "Seminar in Sound Studies" at RPI, which you can read about here. When recording, consider what you're recording, is it a person? an animal? maybe a rock. Always ask for permission if it is a human, and pay attention to their reactions, even if they agree to be recorded. If you are making art with pieces of others, consider if these pieces agree with your message.