Added shallow search for data.table in tables()

[manmita]

Closes #2606

added arg depth = 1L to tables() one for shallow search
if depth is 0 then its the data.table
if depth is 1, we loop through list-like objects using is.list and which are not data.table or data.frame
if depth > 1, we throw error

added name for the nested list found parent[[1]] or parent$child
pre-allocating info to avoid reallocation cost

[manmita]

Hello,

I created a new PR in replacement of #7568

Reasons: There was some git issue there and the merge became too complex and I changed the algo because I didnt know previously that rbind or cbind would cost for re-allocation

The current PR considers that part and avoids appends

Previous PR : creating seperate data.table called info and rbind at the end
This PR: pre-allocates for a total-sized data.table and fills the info

[manmita]

In reply to previous comment of @jangorecki

An example of when this new feature could be useful?

To support lists which occur due to split.data.table or fread like the following

list(data.table(a = 1, b = 4:6)),
      data.table(a = 2, b = 7:10))

The original code supported data.table() top level and this code adds support for list(data.table) if the arg shallow_search = TRUE

[manmita]

Example of the original code and the new feature is as follows

> A = list(data.table(a = 1, b = 4:6),
      data.table(a = 2, b = 7:10))
> B = list(data.table(a = 1, b = 4:6), 1:5)
> C = data.table(a = 1, b = 4:6)
> tables()
   NAME NROW NCOL MB COLS    KEY
1:    C    3    2  0  a,b [NULL]
Total: 0MB using type_size
> tables(shallow_search = TRUE)
     NAME NROW NCOL MB COLS    KEY
1: A[[1]]    3    2  0  a,b [NULL]
2: A[[2]]    4    2  0  a,b [NULL]
3: B[[1]]    3    2  0  a,b [NULL]
4:      C    3    2  0  a,b [NULL]
Total: 0MB using type_size
> D = list(d = data.table(a = 1, b = 4:6), x = 1:5)
> tables(shallow_search = TRUE)
     NAME NROW NCOL MB COLS    KEY
1: A[[1]]    3    2  0  a,b [NULL]
2: A[[2]]    4    2  0  a,b [NULL]
3: B[[1]]    3    2  0  a,b [NULL]
4:      C    3    2  0  a,b [NULL]
5:    D$d    3    2  0  a,b [NULL]
Total: 0MB using type_size

tables() work same as before and tables(shallow_search = TRUE) searches 1 level

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.03%. Comparing base (1bd88cb) to head (c65ff92).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #7580   +/-   ##
=======================================
  Coverage   99.02%   99.03%           
=======================================
  Files          87       87           
  Lines       16896    16937   +41     
=======================================
+ Hits        16732    16773   +41     
  Misses        164      164           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

No obvious timing issues in HEAD=feat/adding_list_search_to_tables

Generated via commit c65ff92

Download link for the artifact containing the test results: ↓ atime-results.zip

Task	Duration
R setup and installing dependencies	3 minutes and 1 seconds
Installing different package versions	22 seconds
Running and plotting the test cases	4 minutes and 2 seconds

[MichaelChirico]

prefer saving the output to re-running tables(env = xenv2, shallow_search = TRUE) many times.

alternatively, just do one test like

test(2360.1, tables(env = xenv2, shallow_search = TRUE)[, .(NAME, NROW, NCOL)], data.table(...))

[manmita]

making it one test

[MichaelChirico]

nit: I would move setindex() closer to the index=TRUE tests

[MichaelChirico]

this comment doesn't make sense outside the context of this PR, "the original code path" will be a relic within a few months

[MichaelChirico]

I am thinking the best way to go about this is actually something like either depth=0 or recursive=FALSE. Then both cases share the same logic, except that the default cuts out after the shallow search.

If the code to do a recursive walk is proving intimidating, we can do depth=0, and this PR can support depth=1 and error for depth>1 "not yet supported" and leave it for future work.

WDYT?

[manmita]

yeah we can do a depth = 0, 1 and error at depth >1 for this PR

[MichaelChirico]

ah, sorry, the "correct" way to make that lint go away is to use stopf() instead

[MichaelChirico]

Suggested change

	#make a listof size list_index and add wl in it
	# make a list of size list_index and add wl in it