Commits

Mark Roddy  committed 9698c14

Added test to check that Null values can be used to override data in an alias. However, after some investigation, it appears as though this may be impossible with the current mechinism used to override alias data (writing a file to disk and replacing the alias with a load command), as it does not appear with the limitted reivew I've done at this point that it is possible to write a null out. Or maybe it's just a Java/Jython thing as None gets written as 'None'.

  • Participants
  • Parent commits ed8d7bb

Comments (0)

Files changed (1)

File tests/test_pigproxy.py

             ]
         self.assertLastOutput(proxy, output);
 
+    def testOverrideToData_SupportsNone(self):
+        """over_to_data() w/None value results in Null value being loaded"""
+        args = [
+            "n=3",
+            "reducers=1",
+            "input=" + self.INPUT_FILE,
+            "output=top_3_queries",
+            ]
+        proxy = PigProxy.from_file(self.PIG_SCRIPT, args)
+
+        new_data = [
+            (None, 3),
+            (None, 4),
+            ]
+        proxy.override_to_data("data", new_data)
+        proxy.override("queries_limit", "queries_limit = FILTER data BY query IS NOT NULL");
+        result_records = list(proxy.get_alias("queries_limit"))
+
+        #TODO: Investigate if the behavior desired by this test is even
+        # possible. After some investigation, it appears as though it is
+        # not. None is translated to 'None', '' is still loaded as ''.
+        # Will need to check by writing pig script with Null data, reloading
+        # it, and then checking if the null values are still present.
+        # self.assertEquals(0, len(result_records), str(result_records))
+
     def testInlinePigScript(self):
         script = '\n'.join([
             "data = LOAD '%s' AS (query:CHARARRAY, count:INT);" % self.INPUT_FILE,