问题:推荐系统中,用户和物品矩阵进行embedding,但矩阵过大时,超过2G时,会有如下提示
ValueError: Cannot create a tensor proto whose content is larger than 2GB.
出现问题的代码语句是:
self.user_item_embedding = tf.convert_to_tensor(matrix)
matrix是一个numpy的大型矩阵。
解决:
matrix_init = tf.placeholder(tf.float32, shape=(self.shape[0], self.shape[1])) matrixV = tf.Variable(matrix_init) self.user_item_embedding = tf.convert_to_tensor(matrixV)
先构建占位符,在设置变量,并在sess.run时赋予:
elf.sess.run(tf.global_variables_initializer(), feed_dict={matrix_init: matrix})
参考:
https://stackoverflow.com/questions/51470991/create-a-tensor-proto-whose-content-is-larger-than-2gb
https://stackoverflow.com/questions/35394103/initializing-tensorflow-variable-with-an-array-larger-than-2gb