sql及list去重操作

一、sql使用场景1：分页查询任务对应的最后一条审核原因

1、查询主sql：用主表task链接审核记录表resion

SELECT * FROM task t LEFT JOIN resion r ON t.id=r.task_id  LEFT JOIN xxxtable;

2、问题分析：一个任务对应多条审核记录，链接查询会产生笛卡尔积，结果集错误。对sql进行改造

SELECT
	* 
FROM
	task t
	LEFT JOIN (
	SELECT
		r.* 
	FROM
		resion r
		INNER JOIN ( SELECT task_id, max( create_time ) create_time FROM resion GROUP BY task_id ) a ON a.task_id = r.task_id 
	AND a.create_time = r.create_time 
	) e ON e.task_id = t.task_id

3、优化：当同一时间出现两条审核记录时依然避免不了数据集错误，所以可以加跟多的条件来保证附表的唯一性

SELECT
	* 
FROM
	task t
	LEFT JOIN (
	SELECT
		r.* 
	FROM
		(
		SELECT
			a.*,
			count( 1 ) onlyValue 
		FROM
			resion a
			INNER JOIN resion b ON a.task_id = b.task_id 
			AND a.create_time <= b.create_time 
		GROUP BY
			a.resion_id 
		) r 
	WHERE
		r.onlyValue = 1 
	) e ON e.task_id = t.task_id

二、list去重处理

1、采用TreeSet来去重

1.1、两次转换

// 这里是没去重的原始数据
List<TaskInfo> taskList=...;  
// 根据id属性去重
Set<TaskInfo> taskSet = new TreeSet<>(Comparator.comparing(TaskInfo::getId));
taskSet.addAll(taskList);
taskList=new ArrayList<TaskInfo>(taskSet);

1.2、流式转换

 List<TaskInfo> newList = taskList.stream().collect(Collectors
  .collectingAndThen(
   Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(TaskInfo::getId))),
   ArrayList::new));

1.3、多属性联合去重

Set<TaskInfo> taskSet = new TreeSet<>(Comparator.comparing(o -> (o.getId() + "" + o.getDeptId())));
taskSet .addAll(taskList);
taskList=new ArrayList<TaskInfo>(taskSet);

2、采用map处理

2.1、流过滤

List<TaskInfo> taskList = new ArrayList<>();
oldList.stream().filter(distinctByKey(t ->t.getId())).forEach(taskList::add);

static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
 Map<Object,Boolean> seen = new ConcurrentHashMap<>();
 return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
}

2.2、流分组循环取第一个