Firebase Android offline performance
When storing around 5000 child nodes under a single node, Firebase is very slow to initialize when using the offline feature. It takes about 30 seconds to execute the first query. Once initialized, executing subsequent queries (such as listing the top 25 children) will take less than a second.
I am using the following property to enable offline functionality: Firebase.getDefaultConfig(). setPersistenceEnabled(true); firebase.keepSynced(true);
My structure looks like this:
<root>
|-my-app-name
|-<uid>
|-node
|-sub node 1
|-...
|-sub node 5000
Keep Sync is set on the <uid>node . Child nodes are displayed in the Recycler View. Preferably, I'd like to list everything (instead of 25 per page), but I know that's not possible because there is no Cursor-like mechanism available for Firebase (Android provides for SQLite).
This is by design, do I modify the data structure? Or is there some other way to reduce initialization time?
I've provided some logging below. As you can see, a lot of garbage collection is going on. Does Firebase evaluate the entire database when initializing?
Thanks! Nils
04-01 15:59:12.029 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 43005(1717KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 31MB/32MB, paused 5.674ms total 57.402ms
04-01 15:59:13.415 2222-2240/abcdef W/art: Suspending all threads took: 6.600ms
04-01 15:59:13.424 2222-2245/abcdef W/art: Suspending all threads took: 9.339ms
04-01 15:59:13.433 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 7097(281KB) AllocSpace objects, 0(0B) LOS objects, 0% free, 32MB/32MB, paused 11.175ms total 27.105ms
04-01 15:59:13.821 2222-2245/abcdef I/art: Background partial concurrent mark sweep GC freed 101674(5MB) AllocSpace objects, 18(530KB) LOS objects, 35% free, 28MB/44MB, paused 3.400ms total 152.664ms
04-01 15:59:15.107 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 394024(15MB) AllocSpace objects, 0(0B) LOS objects, 20% free, 30MB/38MB, paused 1.865ms total 152.182ms
04-01 15:59:15.817 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 218328(8MB) AllocSpace objects, 0(0B) LOS objects, 19% free, 31MB/38MB, paused 1.711ms total 112.325ms
04-01 15:59:16.451 2222-2240/abcdef W/art: Suspending all threads took: 27.786ms
04-01 15:59:16.465 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 190591(7MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 31MB/38MB, paused 1.832ms total 107.416ms
04-01 15:59:16.472 2222-2245/abcdef W/art: Suspending all threads took: 6.823ms
04-01 15:59:17.084 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 178714(6MB) AllocSpace objects, 0(0B) LOS objects, 15% free, 32MB/38MB, paused 1.717ms total 105.529ms
04-01 15:59:17.629 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 163584(6MB) AllocSpace objects, 0(0B) LOS objects, 14% free, 33MB/38MB, paused 1.743ms total 110.764ms
04-01 15:59:18.941 2222-2240/abcdef W/art: Suspending all threads took: 5.078ms
04-01 15:59:19.691 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 95627(3MB) AllocSpace objects, 0(0B) LOS objects, 8% free, 35MB/38MB, paused 7.190ms total 86.171ms
04-01 15:59:19.961 2222-2240/abcdef W/art: Suspending all threads took: 18.208ms
04-01 15:59:20.965 2222-2245/abcdef W/art: Suspending all threads took: 5.254ms
04-01 15:59:20.990 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 55899(2MB) AllocSpace objects, 0(0B) LOS objects, 5% free, 36MB/38MB, paused 6.799ms total 66.923ms
04-01 15:59:22.495 2222-2240/abcdef W/art: Suspending all threads took: 45.180ms
04-01 15:59:22.509 2222-2245/abcdef W/art: Suspending all threads took: 14.254ms
04-01 15:59:22.562 2222-2245/abcdef I/art: Background partial concurrent mark sweep GC freed 198174(6MB) AllocSpace objects, 3(487KB) LOS objects, 32% free, 33MB/49MB, paused 16.949ms total 215.369ms
04-01 15:59:23.811 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 392437(15MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 35MB/43MB, paused 1.936ms total 168.222ms
04-01 15:59:24.480 2222-2240/abcdef W/art: Suspending all threads took: 22.464ms
04-01 15:59:24.497 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 227043(8MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 35MB/43MB, paused 1.723ms total 117.855ms
04-01 15:59:25.173 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 203910(7MB) AllocSpace objects, 0(0B) LOS objects, 16% free, 36MB/43MB, paused 1.694ms total 112.618ms
04-01 15:59:25.181 2222-2245/abcdef W/art: Suspending all threads took: 7.301ms
04-01 15:59:25.784 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 185627(7MB) AllocSpace objects, 0(0B) LOS objects, 14% free, 37MB/43MB, paused 1.719ms total 115.362ms
04-01 15:59:26.345 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 167066(6MB) AllocSpace objects, 0(0B) LOS objects, 13% free, 37MB/43MB, paused 1.651ms total 106.055ms
04-01 15:59:26.865 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 154535(6MB) AllocSpace objects, 0(0B) LOS objects, 11% free, 38MB/43MB, paused 1.644ms total 104.888ms
04-01 15:59:28.357 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 151375(5MB) AllocSpace objects, 33(671KB) LOS objects, 9% free, 39MB/43MB, paused 2.740ms total 104.176ms
04-01 15:59:29.006 2222-2240/abcdef W/art: Suspending all threads took: 19.232ms
04-01 15:59:29.060 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 133554(5MB) AllocSpace objects, 29(580KB) LOS objects, 10% free, 39MB/43MB, paused 1.563ms total 100.220ms
04-01 15:59:30.173 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 131062(4MB) AllocSpace objects, 31(637KB) LOS objects, 9% free, 39MB/43MB, paused 1.653ms total 102.705ms
04-01 15:59:31.245 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 122085(4MB) AllocSpace objects, 26(522KB) LOS objects, 8% free, 39MB/43MB, paused 2.380ms total 100.776ms
04-01 15:59:32.024 2222-2240/abcdef W/art: Suspending all threads took: 20.662ms
PS: This is a cross post : https://groups.google.com/forum/#!topic/firebase-talk/migEAwv26ns
So sharding the data so that a root node has at most 200 children seems to be the answer at the moment. I set .keepSynced(true) on the shard, which resulted in better performance.
To display a list of shards in a single recycler view, I created a FirebaseArrays class which is a collection of FirebaseArrays that aggregates multiple arrays into a single observable collection.
https://gist.github.com/ndefeijter/2191f8a43ce903c5d9ea69f02c7ee7e9
I also modified FirebaseRecyclerAdapter to use FirebaseArrays as the underlying data structure instead of a single FirebaseArray . The interface was extended with some methods to add additional Firebase paths (i.e. shards).
https://gist.github.com/ndefeijter/d98eb5f643b5faf5476b8a611de912c1
These paths are added on the "load more" event (e.g. in the case of endless scrolling).
private void loadMore() {
final View view = getView();
if (null != view) {
final RecyclerView recyclerView = (RecyclerView) view.findViewById(R.id.recycler_view);
final FirebaseRecyclerAdapter2<Visit, VisitViewHolder> adapter = (FirebaseRecyclerAdapter2<Visit, VisitViewHolder>) recyclerView.getAdapter();
adapter.addQuery(nextQuery());
}
}