Skip to content

[oneshot][s3] SS-50 Add cases to COPY FROM s3 testdrive #34849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
patrickwwbutler merged 1 commit into from
Jan 29, 2026
+29 −1
Merged

[oneshot][s3] SS-50 Add cases to COPY FROM s3 testdrive #34849

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion test/testdrive/copy-from-s3-minio.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,48 @@ $ set-max-tries max-tries=1
$ postgres-execute connection=postgres://mz_system:materialize@${testdrive.materialize-internal-sql-addr}
ALTER SYSTEM SET enable_copy_from_remote = true;


# Prepare the table we want to COPY INTO.
> CREATE TABLE t1 (a text, b text);
> CREATE TABLE t1_num (a text, b numeric);
> CREATE TABLE t1_bad_cast (a jsonb, b int);
> CREATE TABLE t1_single_col (a text);
> CREATE TABLE t1_not_null (a text, b int, c float NOT NULL);
> CREATE TABLE t1_fun_house (y int, z text, c float NOT NULL DEFAULT 1.11, d bigint);

$ s3-file-upload bucket=copytos3 key=csv/1.csv repeat=2
none,100

$ s3-set-presigned-url bucket=copytos3 key=csv/1.csv var-name=1_csv_url

# everything imports as text
> COPY INTO t1 FROM '${1_csv_url}' (FORMAT CSV);

> SELECT * FROM t1;
none 100
none 100

# columns can be cast to types
> COPY INTO t1_num FROM '${1_csv_url}' (FORMAT CSV);
> SELECT *, pg_typeof(b) FROM t1_num;
none 100 numeric
none 100 numeric

# FYI: casting a number to json/jsonb, e.g. '1'::jsonb, is valid, casting alphabetic string is not
! COPY INTO t1_bad_cast FROM '${1_csv_url}' (FORMAT CSV);
contains:failed to decode Row

! COPY INTO t1_single_col FROM '${1_csv_url}' (FORMAT CSV);
contains:wrong number of columns

# disabled as MZ panics database-issues#9886
Copy link
Contributor Author

@patrickwwbutler patrickwwbutler Jan 29, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is enabled in a followup: #34862

# ! COPY INTO t1_not_null FROM '${1_csv_url}' (FORMAT CSV);
# contains:some error that's not a panic

> COPY INTO t1_fun_house (z, y) FROM '${1_csv_url}' (FORMAT CSV);
> SELECT * from t1_fun_house;
100 none 1.11 <null>
100 none 1.11 <null>

# gzip Compression.

$ s3-file-upload bucket=copytos3 key=csv/2.csv.gz repeat=2 compression=gzip
Expand Down