How to remove attached elements using delete

I have a simple code with 2 grids. A div contains a button called "Add". When clicked, additional buttons will be added in the second div. When you click any button in the second div it may delete itself. As you can see in the code below, I am using append to add the button in the second div. That's my problem. This code works if the delete button is already in a div. (Indicates that it will be deleted when clicked). However, the attached button does not delete itself. How to add additional button to delete itself after click?

<title>AutoSuggestion using  Ajax, Jquery and PHP</title>

<script src="js/jquery-1.11.1.min.js" ></script>
<script type="text/javascript">

$(document).ready(function(){


            $('#add').click(function(){
                var return_value = "appended button (click to delete me)";
                $('.seconddiv').append('<button class ="remove">'+return_value+'</button>'); 


            });
            $( '.remove' ).click(function() {
              $( this ).remove();

            });

});


</script>

<div class="firstdiv"><button id ="add" >add</button></div>
<div class="seconddiv"><button class ="remove" >click to delete me</button></div>