Import data from google spreadsheet #88

Prakriti-nith · 2018-04-07T20:53:19Z

#87
This PR adds a feature to retrieve data from google spreadsheets.

Enable rubocop MethodLength in initialize method

Prakriti-nith · 2018-04-13T11:49:13Z

Once the code is reviewed, I will add the tests for the related methods.

Shekharrajak · 2018-04-13T15:08:18Z

Writing testcases before writing lines of code is good practice. So testcases must be present for each line of code (test coverage 100%).

Shekharrajak · 2018-04-13T15:09:43Z

spec/adapters/googlecharts_spec.rb

@@ -30,6 +30,14 @@
      ['Lg. Magellanic Cloud', 50000, 120.9],
      ['Bootes I', 60000, 1223.1]
    ]
+    @query_string = 'SELECT A, H, O, Q, R, U LIMIT 5 OFFSET 8'


Please write examples in dummy_iruby and share the link ( like).

I am expecting notebook link like this .

Here is the link to view the examples.
I am getting the outputs for all the jupyter notebook cells locally but I can't understand why they are not visible online.

Did you try in incognito mode ?

Yes, I tried the examples in incognito mode. Running each cell individually is showing the correct output but after I save the file and open it again, then only the last output is visible.

Shekharrajak · 2018-04-13T15:10:13Z

lib/daru/view/adapters/googlecharts.rb

        def init_script
          GoogleVisualr.init_script
        end

        def generate_body(plot)
-          plot.to_html
+          plot.to_html(@data)


Any alternate way, you can think of ?

Shekharrajak · 2018-04-13T15:11:06Z

lib/daru/view/adapters/googlecharts/display.rb

+    # Parameters:
+    #  *data              [Required] The URL of the spreadsheet in a specified format. Query string can be appended
+    #                               to retrieve the data accordingly.
+    #  *div_id            [Required] The ID of the DIV element that the Google Chart should be rendered in.


line size ?

Shekharrajak

Please add few more examples showing how different kind of data can be imported using google spreadsheet in iruby notebook. Also display the dataframe or google datatable for the data imported from google spreadsheet in iruby examples.

I hope you have added testcases covering all the lines.

Shekharrajak · 2018-04-19T10:22:28Z

lib/daru/view/adapters/googlecharts.rb

+            data.table
+          elsif data.is_a?(GoogleVisualr::DataTable)
+            data
+          elsif data.is_a?(String)


Check whether it is a valid URL or not. If not rescue the Exception + document this enhancement.

Where should I write the documentation?

Prakriti-nith · 2018-05-29T20:18:02Z

I have resolved the problem of only last chart visibility. We need to have the separate handle_query_response function for each query (refer).
For drawing the chart when data is imported from Google spreadsheet, we are not using google_visualr as we can't feed the data into the table. The query automatically generates the table when var data_table = response.getDataTable(); js code is run.
I have also improved the documentation according to the yard and added some more tests.
Updated examples: link

Prakriti-nith · 2018-06-03T22:29:20Z

@Shekharrajak ping!

Shekharrajak · 2018-06-10T07:04:51Z

lib/daru/view/adapters/googlecharts.rb

@@ -97,7 +121,7 @@ def extract_chart_type(options)
          chart_type = chart_type.to_s.capitalize
          chart_type = 'SteppedArea' if chart_type == 'Steppedarea'
          chart_type = 'TreeMap' if chart_type == 'Treemap'
-          direct_name = %w[Map Histogram TreeMap Timeline Gauge]
+          direct_name = %w[Map Histogram TreeMap Timeline Gauge Table]


It is only for chart; not for table. Where do you need this ?

Shekharrajak · 2018-06-10T07:05:29Z

lib/daru/view/adapters/googlecharts.rb

-            else
-              add_data_in_table(data)
-            end
+          @table = get_table(data) unless data.is_a?(String)


What will happen if string is not a URL?

What will be plot.table when we have URL?

plot.table will contain the empty table as the DataTable is generated in query response here and we can not retrieve the data from google spreadsheet.

We might be able to retrieve the data from the URL provided. It is because the URL provides the access to JSON data which can then be stored in a Daru::DataFrame.

We can get the table , as here.

Shekharrajak · 2018-06-10T07:12:55Z

spec/adapters/googlecharts/base_chart_spec.rb

+describe GoogleVisualr::BaseChart do
+  before { Daru::View.plotting_library = :googlecharts }
+  before(:each) do
+    @query_string = 'SELECT A, H, O, Q, R, U LIMIT 5 OFFSET 8'


We must use let instead of @.
Please follow better specs.

Shekharrajak · 2018-06-10T07:18:43Z

lib/daru/view/adapters/googlecharts/display.rb

+    # @param dom [String] The ID of the DIV element that the Google
+    #   Chart should be rendered in
+    # @return [String] js code to render the chart
+    def get_html(dom)


I think, it is same as to_js method.

Actually, I was going to ask you about this. I wanted to know why are we generating the googlecharts' code with script tag (by calling to_js method) and without script tag (by calling load_js and draw_js) separately in display.rb. We have already defined script tags in the template so we'll definitely be generating the script without the script tags (using load_js and draw_js)?
to_js method is never actually called.

Shekharrajak · 2018-06-10T07:20:35Z

lib/daru/view/adapters/googlecharts.rb

+        # @param options [Hash] Various options provided by the user to
+        #   incorporate in google charts
+        # @return [GoogleVisualr::Interactive] Returns the chart object based
+        #   on the chart_type


Thanks for writing this. Please write few examples as well for init and init_table.

Prakriti-nith · 2018-06-14T06:43:48Z

@Shekharrajak please review the changes.

Shekharrajak

I hope specs are covering all the methods/lines you have written.

Shekharrajak · 2018-06-14T17:19:03Z

lib/daru/view/adapters/googlecharts.rb

+        #          '_7zOFDcg8Wm8Xv29_8PWuuW15qmAE/gviz/tq?gid=0&headers=1&tq='
+        #   query = 'SELECT A, H, O, Q, R, U LIMIT 5 OFFSET 8'
+        #   data << query
+        #   chart = Daru::View::Table.new(data)


IRuby notebook example for this?

I have already added example for this. I have further added an example for the previous one. Link.

Shekharrajak · 2018-06-14T17:20:26Z

lib/daru/view/adapters/googlecharts.rb

-          add_data_in_table(data)
+          @table.data = data
+          # When data is the URL of the spreadsheet then plot.table will contain the empty
+          #   table as the DataTable is generated in query response in js and we can not


Line length < 80 ,please.

Shekharrajak · 2018-06-14T17:22:53Z

spec/adapters/googlecharts_spec.rb

@@ -30,6 +30,14 @@
      ['Lg. Magellanic Cloud', 50000, 120.9],
      ['Bootes I', 60000, 1223.1]
    ]
+    @query_string = 'SELECT A, H, O, Q, R, U LIMIT 5 OFFSET 8'
+    @data_spreadsheet = 'https://docs.google.com/spreadsheets/d/1XWJLkAwch'\


I know there are some cases where @ is used (which is not correct), please use let instead.

Shekharrajak

Looks good to me.

Shekharrajak · 2018-06-16T09:47:37Z

lib/daru/view/adapters/googlecharts.rb

-            else
-              add_data_in_table(data)
-            end
+          @table = get_table(data) unless data.is_a?(String)


We can get the table , as here.

Shekharrajak · 2018-06-16T10:13:34Z

Thanks @Prakriti-nith , for writing in wiki page

Import data from google spreadsheet

a59968d

Enable rubocop MethodLength in initialize method

Prakriti-nith force-pushed the import_spreadsheet branch from d8d69f7 to a59968d Compare April 12, 2018 20:17

Fix the testcases

10c4d11

Prakriti-nith force-pushed the import_spreadsheet branch from 2bbe578 to 10c4d11 Compare April 12, 2018 20:55

Shekharrajak requested changes Apr 13, 2018

View reviewed changes

Add tests and examples

f1e6137

Shekharrajak reviewed Apr 19, 2018

View reviewed changes

Prakriti-nith added 3 commits April 22, 2018 18:58

Add more examples and support for google datatables

d2bb96b

Added documentation

1212bab

Resolved error of only last chart visibility

c495929

Prakriti-nith force-pushed the import_spreadsheet branch from 3d8b87e to c495929 Compare May 29, 2018 16:20

Prakriti-nith added 2 commits May 30, 2018 00:50

Improved documentation as per yard

3c4dcbd

Added more specs

7e49295

Removed rubocop errors

71052e0

Prakriti-nith added 3 commits June 4, 2018 20:14

Creater base_chart.rb file in googlecharts

7b8910d

Added more tests

ca52ea9

Proper alignment in web pages

be55df7

Shekharrajak reviewed Jun 10, 2018

View reviewed changes

Validate URL

b0ac922

Prakriti-nith force-pushed the import_spreadsheet branch from fef20e1 to b0ac922 Compare June 11, 2018 02:52

Updated documentation

8a03874

Prakriti-nith force-pushed the import_spreadsheet branch from d9662f3 to 8a03874 Compare June 11, 2018 21:16

Added more tests and examples

3f8d29b

Shekharrajak reviewed Jun 14, 2018

View reviewed changes

Refactoring according to comments

d655555

Shekharrajak approved these changes Jun 16, 2018

View reviewed changes

Shekharrajak merged commit 2db0024 into SciRuby:master Jun 16, 2018

Import data from google spreadsheet #88

Import data from google spreadsheet #88

Uh oh!

Conversation

Prakriti-nith commented Apr 7, 2018

Uh oh!

Prakriti-nith commented Apr 13, 2018

Uh oh!

Shekharrajak commented Apr 13, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Shekharrajak left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Prakriti-nith commented May 29, 2018

Uh oh!

Prakriti-nith commented Jun 3, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Prakriti-nith commented Jun 14, 2018

Uh oh!

Shekharrajak left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Shekharrajak left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Shekharrajak commented Jun 16, 2018

Uh oh!

Uh oh!