当前位置:网站首页>Error when creating dataset with mindscore
Error when creating dataset with mindscore
2022-07-25 19:41:00 【Xiaole happy】
Creating a dataset by yourself will cause errors
【 Operation steps & Problem phenomenon 】
I want to create a super-resolution dataset , This data set has two columns ,data and target,data Is a low resolution image ,target High resolution image . My idea is to create a data and target Image like data set , And then to data Reduce the image in , But an error is reported in the process of creating the dataset
My code
from os import listdir
from os.path import join
import os
from PIL import Image
import random
import numpy as np
import mindspore.dataset as ds
import mindspore.dataset.vision.py_transforms as py_vision
import mindspore.dataset.transforms.py_transforms as py_transforms
def is_image_file(filename):
return any(filename.endswith(extension) for extension in ['.png', '.jpg', '.jpeg', '.PNG', '.JPG', '.JPEG'])
def make_dataset(dir_path, max_dataset_size=float("inf")):
"""Return image list in dir."""
images = []
assert os.path.isdir(dir_path), '%s is not a valid directory' % dir_path
for root, _, fnames in sorted(os.walk(dir_path)):
for fname in fnames:
if is_image_file(fname):
path = os.path.join(root, fname)
images.append(path)
return images[:min(max_dataset_size, len(images))]
class MakeDataset:
def __init__(self, dataset_dir, max_dataset_size=float("inf")):
self.dir_A = os.path.join(dataset_dir,'data')
self.dir_B = os.path.join(dataset_dir,'target')
self.A_paths = sorted(make_dataset(self.dir_A, max_dataset_size))
self.B_paths = sorted(make_dataset(self.dir_B, max_dataset_size))
self.A_size = len(self.A_paths)
self.B_size = len(self.B_paths)
def __getitem__(self, index):
A_path = self.A_paths[index % self.A_size]
B_path = self.B_paths[index % self.B_size]
A_img = np.array(Image.open(A_path).convert('RGB'))
B_img = np.array(Image.open(B_path).convert('RGB'))
print(A_img.shape)
print(B_img.shape)
return A_img,B_img
def __len__(self):
return self.A_size
dataset=MakeDataset(dataset_dir="/data/xjtu/wys/image/")
DS=ds.GeneratorDataset(dataset,column_names=["LR,HR"])
for data in DS.create_dict_iterator():
print(data["LR"].shape)Error message
Traceback (most recent call last):
File "/data/xjtu/wys/SRGAN/dataset.py", line 49, in <module>
for data in DS.create_dict_iterator():
File "/root/archiconda3/envs/xjtu/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 263, in __next__
data = self.get_next()
File "/root/archiconda3/envs/xjtu/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 344, in get_next
return {k: Tensor(v.as_array()) for k, v in self.depipeline.GetNextAsMap().items()}
RuntimeError: Thread ID 281472224702944 Exception thrown from PyFunc. Invalid parameter, Generator should return same number of numpy arrays as specified in column names.
Line of code : 122
File : /home/jenkins/agent-working-dir/workspace/Compile_Ascend_ARM_EulerOS/mindspore/mindspore/ccsrc/minddata/dataset/engine/datasetops/source/generator_op.ccReport errors Generator should return same number of numpy arrays as specified in column names., Description means that the returned field does not match the number of specified column names .
I see in the code :class MakeDataset: -> def __getitem__(self, index): -> return A_img,B_img Two fields are returned .
however DS=ds.GeneratorDataset(dataset,column_names=["LR,HR"]) As defined in column_names Need to change to column_names=["LR", "HR"], Expressed as two fields .
边栏推荐
- KCon 2022 亮点及议程大揭秘!
- Selenium runs slowly - speed up by setting selenium load policy
- Mutual conversion of camera internal parameter matrix K and FOV
- [wp]ctfshow-web getting started - Explosion
- [CSAPP Practice Problem 2.32] tsub_ OK (int x, int y) judge whether complement subtraction overflows
- Beihang and other "deep learning event extraction" literature review paper, 27 page PDF describes the current trend
- 哈希无向图可视化
- 一元函数积分学_分部积分法
- Gbase 8s UDR memory management_ 01_ mi_ alloc
- Old wine in new bottles -- sample analysis of recent apt32 (sea Lotus) organizational attacks
猜你喜欢
随机推荐
微信小程序开发之网络数据请求
GBASE 8s UDR内存管理_03_mi_realloc
Shopping guide for high-end flagship projectors: dangbei X3 pro and dangbei F5 are more immersive!
浅谈接口加密
Scala foundation [set 01]
相机内参矩阵K和fov的相互转换
binarySearch基础二分查找
Js分页插件支持表格、列表、文本、图像
How to get started quickly in software testing
Imperial cms7.5 imitation "question and answer library" question and answer learning platform website source code with mobile version
AAAI 2022 | GAN的结构有“指纹”吗?从伪造图像溯源生成网络结构
Day7: ordered binary tree (binary search tree)
微信小程序10-微搭模板
由一个蓝桥杯基础题报时助手而引出的常见误区
TFIDF examples and explanations
What is idealism
C merge set
Binarysearch basic binary search
微信小程序开发之WXSS模板样式与WXS脚本语言
VMware 虚拟机下载、安装和使用教程








