Duplicate and Remove Table Rows through Jquery

So I have managed to make it so you are able to add and remove table rows by clicking an icon on each table row.

My problem is though, let's say you add two (2) rows and then remove one (1) Then change your mind and add one (1) row to the table again. Well instead of getting one (1) row you get two (2). For some reason jQuery is bringing back that first row you removed plus the new one you just requested.

I opened up Google Chrome Inspector and did see that if you remove the row, it does in fact disappear from markup but yet still comes back when you add a new one.

My table is laid out like so:

<table class="table taskTable" style="text-align:center;">
<thead>
    <tr>
        <th>Task Name</th>

        <th>Est. Task Completion</th>

        <th>Est. Task Hours</th>

        <th>&nbsp;</th>
    </tr>
</thead>

<tbody>
    <tr>
        <td><input class="form-control" name="task_name[]" type="text"
        value=""></td>

        <td><input class="form-control" name="task_done[]" type="text"
        value=""></td>

        <td><input class="form-control" name="task_hour[]" type="text"
        value=""></td>

        <td style="vertical-align:middle; text-align:center;">
            <a class="removeTaskRow fa fa-trash-o" href="#" style=
            "color: #d9534f; font-style: italic; margin-right: 10px" title=
            "Remove Task"></a> <a class="addTaskRow fa fa-plus" href="#"
            style="color: #5cb85c; font-style: italic" title=
            "New Task"></a>
        </td>
    </tr>
</tbody>

I then have the following Javascript for adding and removing the table rows.

$(".addTaskRow").click(function(){
    addTableRow($('.taskTable tbody'));
});

function addTaskRow(){
    addTableRow($('.taskTable tbody'));
}

function removeTaskRow(){ 
    var par = $(this).parent().parent(); 
    par.remove(); 
};

function addTableRow(table){
    $(table).append(
        "<tr>"+
        "<td><input type='text' class='form-control' /></td>"+
        "<td><input type='text' class='form-control' /></td>"+
        "<td><input type='text' class='form-control' /></td>"+
        "<td style='vertical-align:middle; text-align:center;'><a href='#' style='margin-right: 10px; color: #d9534f' class='removeTaskRow' title='Remove Task'><i class='fa fa-trash-o'></i></a><a href='#' style='color: #5cb85c;' class='addTaskRow' title='New Task'><i class='fa fa-plus'></i></a></td>"+
        "</tr>"
    );

    $(".addTaskRow").bind("click", addTaskRow);
    $(".removeTaskRow").bind("click", removeTaskRow);
}

Why would jQuery be bringing back the previously deleted rows when you add a new one?

I have created a JSFiddle for you to see exactly what I am talking about. Try creating some new rows, deleting some, and then adding some back. JSFiddle: http://jsfiddle.net/8sj7n1h1/