Android GridView unpredictable behavior

I am finally confused with a problem related to the Android GridView's. I am sorry for a little bit long description, but I have not a lot of time to optimize my question.

Here is what I want to get. There should be a GridView with (8x10) elements. Each element has an initially generated random color from 5 supported color set. (I have used Android View type for an element and just set a background color).

Every time when user clicks on an element it should be "deleted" and all the elements above it should be moved down to fill the vacant space. ( I am taking the color id from the element above and set it to the current element and do so till reaching the very top element, then randomly generating new color for the top element of the column).

After all of this stuff I just call the function notifyDataSetChanged() on my adapter which were used for the GridView.

The problem is that the behavior is unpredictable. On some devices it works on some devices it updates all the elements and all elements are changed. Sometimes I see that there were no color set which seems was related to the optimization and seems I found a solution...

So, Do anybody faced with some kind of issues? How do you reached a stable (expected) behavior ? May be you can suggest some other (easier) solution or some related documentations/examples/lessons I have already read almost all what Google brings about the GridLayouts.

I am kindly appreciate the time you have spent on my question! Thanks, Arsen

Updated:

Here is some part of codes which I am using: sorry for not publishing some details.

in layout.xml

 <GridView
    class="<name>" 
android:id="@+id/gridView1"
android:listSelector="@null"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:layout_margin="0dp"
android:layout_alignParentBottom="true"
android:horizontalSpacing="0dp"
android:numColumns="8"
android:padding="0dp"
android:stretchMode="columnWidth"
android:cacheColorHint="@android:color/transparent"
android:verticalSpacing="0dp" >
 </GridView>

in layout.simple_grid_view_element:

<View xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/box"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:gravity="center_vertical"
/>

in ImageAdapder which extends BaseAdapter:

 ...
 @Override
 public View getView(int position, View convertView, ViewGroup parent) {
View v = convertView;
if (convertView == null) {  // for the first time initialize each element.
      LayoutInflater Inf = (LayoutInflater) mContext.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
      convertView = Inf.inflate(R.layout.simple_grid_view_element, parent, false);
      v = (ImageView) convertView.findViewById(R.id.box);
      WindowManager wm = (WindowManager) mContext.getSystemService(Context.WINDOW_SERVICE);
      Display display = wm.getDefaultDisplay();
      int width = display.getWidth();
      int height = display.getHeight() - 40;
      v.setLayoutParams(new GridView.LayoutParams(((int)(width)/(mEngine.ColumnNumber)), ((int)((height)/mEngine.RowNumber) - 5)));
          v.setBackgroundResource( mEngine.generate_color_based_on_box_color(mEngine.TABLE.get(position)));
    }
    return v;
 }

in MainActivity:

....
GridView gridview = (GridView) findViewById(R.id.gridView1);
gridview.setAdapter(mEngine.getAdapter());
gridview.setVerticalScrollBarEnabled(false);
gridview.setOnItemClickListener(this);

....
@Override
public void onItemClick(AdapterView<?> parent, View view, int position,
        long id) {
mEngine.RespondOnClick(parent, position);
}

in Engine class I am just changing the background color of the item with given position taking from onItemClick function: This function uses a helper array TABLE which is actually for keeping the color indices because View.getBackground() function returns huge value which is not works correctly because JAVA has not unsigned types... (but this is already resolved). I have debugged the application and all the changes in TABLE array are doing correctly.

...
private ImageAdapter mAdapter;
...
Engine(Context mContext) {
    Columns = new ArrayList<ArrayList<Integer>>(ColumnNumber);
for (int i = 0; i < ColumnNumber; ++i) {
    Columns.add(new ArrayList<Integer>());
}

TABLE = new ArrayList<Integer>(ColumnNumber * RowNumber);
for (int i = 0; i < ColumnNumber * RowNumber; ++i) {
    TABLE.add(i, generateRandomColor());
}

    mAdapter = new ImageAdapter(mContext, this);
}

public void RespondOnClick(AdapterView<?> parent, final int position) {
CollectBoxesMarkedToRemoveByLine();
for (int i = 0; i < Columns.size(); ++i) {
    while (0 != Columns.get(i).size()) {
        int TopDeletedBoxPosition = Columns.get(i).get(0);
        int CurrentRowValue = (TopDeletedBoxPosition / ColumnNumber);
        while (0 < CurrentRowValue) {
            int PrevBoxColor = TABLE.get(TopDeletedBoxPosition - ColumnNumber);
            TABLE.set(TopDeletedBoxPosition, PrevBoxColor);
            this.getBox(TopDeletedBoxPosition).setBackgroundResource(generate_color_based_on_box_color(PrevBoxColor));
            --CurrentRowValue;
            TopDeletedBoxPosition -= ColumnNumber;
        }
        if (0 == CurrentRowValue) {
            int newColor = generateRandomColor();
            TABLE.set(TopDeletedBoxPosition, newColor);
            this.getBox(TopDeletedBoxPosition).setBackgroundResource(generate_color_based_on_box_color(TABLE.get(TopDeletedBoxPosition)));
        }
        Columns.get(i).remove(0);
    }
}
...
mAdapter.notifyDataSetChanged();  // called after RespondOnClick() has done all the replacements.